Batch Compute:Multiple tasks

In Batch Compute, a job can contain multiple tasks which have DAG dependencies between each other.

It means the next task starts to run only when the previous task is finished.

Example 1

Set the job description JSON as follows:

{
  "Name": "my-job",
  "DAG": {
    ...
    "Dependencies": {
      "split": ["count"],
      "count": ["merge"]
    }
  }
}

• After split is finished, count starts to run and then merge starts to run in turn.
• When merge is finished, the whole job finishes.

Example 2

Set the job description JSON as follows:

{
  "Name": "my-job",
  "DAG": {
    ...
    "Dependencies": {
      "split": ["count1","count2"],
      "count1": ["merge"],
      "count2": ["merge"]
    }
  }
}

• After split is finished, count 1 and count 2 start to run simultaneously. After count 1 and count 2 are finished, merge starts to run.

• When merge is finished, the whole job finishes.