After you commit and deploy an auto triggered task for which scheduling properties are configured to the scheduling system, DataWorks automatically generates instances for the auto triggered task based on the scheduling properties. When you configure scheduling properties for an auto triggered task, you can set the Instance Generation Mode parameter to Next Day or Immediately After Deployment. If you set the parameter to Next Day, DataWorks schedules the auto triggered task the next day after the task is deployed to the production environment. If you set the parameter to Immediately After Deployment, DataWorks immediately schedules the auto triggered task after the task is deployed to the production environment. This topic describes the rules for immediate instance generation and how to configure the immediate instance generation feature.
Precautions
After you deploy your auto triggered task for which the Instance Generation Mode parameter is set to Immediately After Deployment to the production environment, we recommend that you go to Operation Center to check whether the configuration meets your business requirements.
Time when auto triggered instances take effect
After an auto triggered task is deployed to the production environment, you can view the latest scheduling dependencies of the auto triggered task on the Cycle Task page in Operation Center. The time when the auto triggered instances generated for the auto triggered task take effect and the time when the scheduling dependencies of the auto triggered instances are updated vary based on the instance generation mode that you select for the auto triggered task.
Time when a change operation takes effect
If you perform a change operation on an auto triggered task during the period from
23:30 to 24:00, the change takes effect on the instances that are generated for the auto triggered task on the day after the next day.Scenarios in which the immediate instance generation feature can be used
We recommend that you configure the immediate instance generation feature for an auto triggered task only when you want to replace the existing instances generated for the auto triggered task based on the new scheduling properties of the auto triggered task. In other cases, we recommend that you do not configure the immediate instance generation feature for an auto triggered task unless unnecessary.
ImportantWhen you change scheduling properties for an auto triggered task that is deployed to the production environment, we recommend that you do not configure the immediate instance generation feature for the task. If you configure the feature for the auto triggered task, the scheduling dependencies of the instances generated for the auto triggered task on the day may be changed or incorrect, or some instances generated for the auto triggered instance on the day are replaced or deleted. As a result, the scheduling dependencies of the instances on the day become complicated.
If you modify the configuration of a deployed auto triggered task and you want the task to be run based on the latest configurations, you can set the Instance Generation Mode parameter to Next Day for the task and backfill data for the task.
If you use the immediate instance generation feature for an auto triggered task, the scheduling dependencies of the instances generated for the task on the next day become normal.
Difference between the time when an auto triggered task is deployed and the time when the immediate instance generation feature takes effect
To prevent instance changes from blocking the running of an auto triggered task for which the immediate instance generation feature is configured, a time difference of 10 minutes exists between the time when the auto triggered task is deployed and the time when the immediate instance generation feature takes effect. DataWorks runs the auto triggered task, generates instances for the auto triggered task based on the latest scheduling properties of the auto triggered task, and schedules the instances 10 minutes after the auto triggered task is deployed.
Effective scope of the immediate instance generation feature
Some changes to the configurations of the auto triggered task may not take effect immediately. For example, if you change the data source that is used by an auto triggered task, configure the immediate instance generation feature for the task, and then deploy the task, the instances generated for the task on the current day are not affected and are still run based on the original data source.
If you want the change to immediately take effect for the auto triggered task, you can backfill data for the task. The data backfill operation is performed based on the latest configurations of the auto triggered task.
Overview
The following table describes the use scenarios of the immediate instance generation feature. This feature has different impacts on the running of instances generated for an auto triggered task and on the scheduling dependencies of the instances in different use scenarios.
Scenario | References |
If you configure the immediate instance generation feature for an auto triggered task that is deployed and deploy the auto triggered task again after the configuration, DataWorks generates instances for the auto triggered task on the current day. Whether the generated instances are run depends on the scheduling time of the auto triggered task. | |
If you change the scheduling time and scheduling frequency of an auto triggered task that is deployed, the scheduling dependencies of the instances generated for the auto triggered task on the current day may be complex. We recommend that you do not configure the immediate instance generation feature for a deployed auto triggered task unless unnecessary. | |
If an auto triggered task and its ancestor task are both newly created tasks and have different instance generation modes, the auto triggered task may become an isolated task. | |
If you change the scheduling time of an auto triggered task and configure the immediate instance generation feature for the auto triggered task, DataWorks automatically adjusts scheduling dependencies for the instances generated for the descendant tasks of the auto triggered task based on the latest scheduling properties of the auto triggered task. The scheduling frequency of an auto triggered task can be daily, monthly, or hourly. |
Scenario 1: Configure the immediate instance generation feature for an auto triggered task that is deployed
If you configure the immediate instance generation feature for an auto triggered task that is deployed and deploy the auto triggered task to the production environment again after the configuration, the running of the instances generated for the auto triggered task depends on the scheduling time of the auto triggered task.
Scheduling time | Description |
The scheduling time of the auto triggered task is later than the time when the auto triggered task is deployed. | DataWorks generates instances for the auto triggered task and schedules the instances based on the scheduling time of the auto triggered task. If the scheduling time of the auto triggered task is later than the time when the auto triggered task is deployed, only instances generated for the auto triggered task on the current day are affected. Whether the existing instances of the auto triggered task are replaced varies based on whether the scheduling time of the task is 10 minutes later than the time when the task is deployed. If the scheduling time of the task is 10 minutes later than the time when the task is deployed, the existing instances are replaced. For more information, see the Precautions section in this topic. |
The scheduling time of the auto triggered task is earlier than the time when the auto triggered task is deployed. | DataWorks generates a dry-run instance for the auto triggered task. The dry-run instance is an expired instance that is generated in real time. If you want the auto triggered task to generate data on the current day, you can backfill data whose data timestamp is the previous day for the auto triggered task. A time difference of 10 minutes exists between the time when the data backfill operation is performed and the time when a data backfill instance is generated for the auto triggered task. For more information, see the Precautions section in this topic. |
Example: If you configure the immediate instance generation feature for an auto triggered task and deploy the auto triggered task to the production environment at 12:00, the immediate instance generation feature takes effect at 12:10 for the auto triggered task.
If the scheduling time of the auto triggered task is later than
12:10, DataWorks runs the auto triggered task and generates instances for the auto triggered task as scheduled.If the scheduling time of the auto triggered task is earlier than
12:10, DataWorks generates a dry-run instance for the auto triggered task. The dry-run instance is an expired instance that is generated in real time.
Scenario 2: Change the scheduling properties of a deployed auto triggered task for which the immediate instance generation feature is configured
If you change the scheduling time and scheduling frequency of an auto triggered task that is deployed to the production environment and configured with the immediate instance generation feature, and deploy the change operation to the production environment, the instances generated for the auto triggered task on the current day before the change and the instances generated for the auto triggered task on the current day after the change may exist at the same time. In this case, the scheduling dependencies of the instances generated for the auto triggered task on the current day may be complex.
The co-existence of the instances generated before and after the change occurs only on the day on which the auto triggered task is deployed. DataWorks generates instances for the auto triggered task the next day based on the latest scheduling properties of the auto triggered task.
Impacts on the instances generated for the auto triggered task vary based on the new scheduling time of the auto triggered task:
The scheduling time of the auto triggered task is later than the time when the auto triggered task is deployed.
DataWorks generates instances based on the latest scheduling properties of the auto triggered task and use the new instances to replace the existing instances.
The scheduling time of the auto triggered task is earlier than the time when the auto triggered task is deployed.
DataWorks retains the instances that are generated before the scheduling time and replaces or deletes the instances that are generated after the scheduling time.
After you change the scheduling time of an auto triggered task and deploy the change operation to the production environment, whether DataWorks generates instances and runs the instances based on the latest scheduling properties of the auto triggered task depends on the new scheduling time of the auto triggered task and the time when you deploy the change operation. For more information, see the Precautions section in this topic.
Scenario 3: Configure different instance generation modes for an auto triggered task and its ancestor task
If an auto triggered task and its ancestor task are both newly created tasks and have different instance generation modes, the auto triggered task may become an isolated task. For example, if you set the Instance Generation Mode parameter to Next Day for the ancestor task and the Instance Generation Mode parameter to Immediately After Deployment for the auto triggered task, the auto triggered task may become an isolated task. An isolated task is not automatically scheduled. If multiple tasks depend on the isolated task, your business may be severely affected.
Scenario 4: Change the scheduling time of an ancestor auto triggered task, which affects the scheduling dependencies of descendant instances
If you change the scheduling time of an auto triggered task and configure the immediate instance generation feature for the auto triggered task, DataWorks automatically adjusts scheduling dependencies for the instances that are generated for the descendant task of the auto triggered task based on the latest scheduling properties of the auto triggered task. The scheduling frequency of an auto triggered task can be daily, monthly, or hourly.
If you change the scheduling time of an auto triggered task in the production environment and configure the immediate instance generation feature for the auto triggered task, DataWorks automatically establishes scheduling dependencies for the instances that are generated for the descendant task of the auto triggered task based on the latest scheduling properties of the auto triggered task. The instances include the instances that are generated after the change of the scheduling time and the instances that are generated before the change of the scheduling time but are not replaced. For more information, see Principles and samples of scheduling configurations in complex dependency scenarios. If you change the scheduling time of an auto triggered task that waits to be deployed and is configured with the immediate instance generation feature, the scheduling dependencies of the instances generated for the descendant task of the auto triggered task after the auto triggered task is deployed are affected.
Examples:
Example 1: The scheduling frequency of an ancestor auto triggered task is changed from every 6 hours to every 8 hours, and the immediate instance generation feature is configured for the auto triggered task.
Example 2: The scheduling frequency of an ancestor auto triggered task is changed from every 6 hours to 16:00 every day, and the immediate instance generation feature is configured for the auto triggered task.
