Skip to content
This repository was archived by the owner on Jan 6, 2018. It is now read-only.
This repository was archived by the owner on Jan 6, 2018. It is now read-only.

oozie coordinator input-event does not work #839

@twmht

Description

@twmht

hi,

I have three coordinators A, B and C.

The coordinator of B and C depends on the output of A. That is, if the output of A is ready, coordinator of B and C will run.

So, I use an input-event to control such dependency.

The structure of coordinator B and C look like

<coordinator-app name="B" frequency="1440" start=${start} end=${end} timezone="UTC" xmlns="uri:oozie:coordinator:0.1">
   <datasets>
      <dataset name="input1" frequency="1440" initial-instance=${start} timezone="UTC">
         <uri-template>hdfs://localhost:9000/tmp/revenue_feed/${YEAR}/${MONTH}/${DAY}</uri-template>
      </dataset>
   </datasets>
   <input-events>
      <data-in name="coordInput1" dataset="input1">
          <instance>${coord:current(0)}</instance>
      </data-in>
   </input-events>
   <action>
      <workflow>
         <app-path>hdfs://localhost:9000/B/workflows</app-path>
      </workflow>
   </action>     
</coordinator-app>

So, if hdfs://localhost:9000/tmp/revenue_feed/${YEAR}/${MONTH}/${DAY}/_SUCCESS is created, the coordinator B and C will be triggered to run their workflow.

the coordinator of A looks like:

<coordinator-app name="B" frequency="1440" start=${start} end=${end} timezone="UTC" xmlns="uri:oozie:coordinator:0.1">
   <action>
      <workflow>
         <app-path>hdfs://localhost:9000/A/workflows</app-path>
      </workflow>
   </action>
</coordinator-app>

its ${start} and ${end} are same as B and C.

The workflow of A will create hdfs://localhost:9000/tmp/revenue_feed/${YEAR}/${MONTH}/${DAY}/_SUCCESS

However, the coordinator of B and C are still waiting for the hdfs://localhost:9000/tmp/revenue_feed/${YEAR}/${MONTH}/${DAY}/_SUCCESS

Even though I use output event of coordinator of A, the workflow of B and C are still wanting for the created input dataset.

<coordinator-app name="B" frequency="1440" start=${start} end=${end} timezone="UTC" xmlns="uri:oozie:coordinator:0.1">
    <datasets>
        <dataset name="input1" frequency="1440" initial-instance=${start} timezone="UTC">
        <uri-template>hdfs://localhost:9000/tmp/revenue_feed/${YEAR}/${MONTH}/${DAY}</uri-template>
        </dataset>
    </datasets>

    <output-events>
        <data-out name="coordOutput1" dataset="output1">
            <instance>${coord:current(0)}</instance>
        </data-out>
    </output-events>
   <action>
      <workflow>
         <app-path>hdfs://localhost:9000/A/workflows</app-path>
      </workflow>
   </action>
</coordinator-app>

If I submit the workflow of A without its coordinator, then the workflow of B and C will be triggered.

I am not sure if something missing in my coordinator of A.

Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions