View auto triggered instances

An auto triggered instance is a snapshot that is taken for an auto triggered task at the time when the task is scheduled to run. You can view the details of and perform the related operations on an auto triggered instance in the list of auto triggered instances or in the directed acyclic graph (DAG) of the instance.

Usage notes

Normal task or instance: A normal task or instance is a task or instance that actually runs code. Normal tasks or instances do not include dry-run tasks or instances and frozen tasks or instances. The dry-run tasks or instances include tasks whose scheduling mode is set to dry-run, instances that are not generated during the period of time in which tasks are scheduled, branch tasks that are not selected, and expired instances that are generated when the scheduled time for running tasks is less than 10 minutes from the point in time when the tasks are committed and deployed.
O&M environment: In a DataWorks workspace in standard mode, you can click the switch icon to the right of Operation Center in the top navigation bar to switch between the development and production environments. Instances cannot be automatically scheduled in Operation Center in the development environment. Nodes cannot be automatically scheduled to generate auto triggered instances on the Cycle Instance page.
Instance running and issue troubleshooting:
- An auto triggered instance can be scheduled to run only when the following conditions are met: Ancestor tasks of the task for which the auto triggered instance is generated are successfully run, the scheduling time of the auto triggered instance has arrived, scheduling resources are sufficient, and the auto triggered instance is not frozen. For more information, see What are the conditions that are required for a node to successfully run?
- If an auto triggered instance fails to run, you can use the ancestor task analysis feature on the Upstream Analysis tab of the DAG page to quickly identify ancestor instances that block the running of the current instance. Then, you can use the intelligent diagnosis feature to diagnose failure causes or related issues of the ancestor instances. The intelligent diagnosis feature can also be used to quickly troubleshoot issues when dependencies between the current instance and ancestor instances are complex. This improves O&M efficiency.

Limits

Editions:
- Only users of DataWorks Professional Edition or a more advanced edition can use the intelligent diagnosis feature. If you use another edition, you can have a trial use of the feature for free. However, we recommend that you upgrade the DataWorks service to DataWorks Professional Edition to use more features. For more information, see Intelligent diagnosis.
- Only users of DataWorks Professional Edition or a more advanced edition can use the aggregation, upstream analysis, and downstream analysis features provided by DAGs. For more information, see Billing of DataWorks advanced editions.
Permissions:
Specific features can be used only by users who are granted O&M permissions. If the entry point of a feature is dimmed or is not displayed, you can check whether you are granted the required O&M permissions on the Workspace Members tab of the Workspace page in SettingCenter. For more information, see Go to the SettingCenter page and Manage permissions on workspace-level services.
Features:
- You cannot manually delete an auto triggered instance. DataWorks deletes an auto triggered instance approximately 30 days after the instance expires. If you no longer need to run an auto triggered instance, you can freeze the instance.
- Instances that are run on the shared resource group for scheduling are retained for one month (30 days), and the operation logs for the instances are retained for one week (7 days).
- Instances that are run on an exclusive resource group for scheduling are retained for one month (30 days), and the operation logs for the instances are also retained for one month (30 days).
- The system clears excess operation logs every day when the size of operation logs generated for the auto triggered instances that finish running exceeds 3 MB.

Precautions

Instances are generated for auto triggered as scheduled. Each generated instance runs the most recent code. If you modify and recommit the code of a task after instances are generated for the task, the instances that are not run will run the most recent code.
If you want to monitor a task for which instances are generated, you must configure monitoring rules for the task first. For more information, see Overview. If a task for which monitoring rules are configured fails to run and you do not receive an alert notification, check whether your mobile phone number and email address are configured on the Alert Contacts page. For more information, see View alert details.
The time at which instances are generated for an auto triggered varies based on the value of the Instance Generation Mode parameter. The valid values of the Instance Generation Mode parameter include Next Day and Immediately After Deployment. For more information, see Modes in which instances take effect.
Note
Tasks that are manually rerun do not trigger alerts generated based on custom rules.

Go to the Auto Triggered Instances page

Go to the Operation Center page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. Then, choose Data Modeling and Development > Operation Center in the left-side navigation pane. On the page that appears, select the desired workspace from the drop-down list and click Go to Operation Center.
In the left-side navigation pane of the Operation Center page, choose Auto Triggered Node O&M > Auto Triggered Instances.
On this page, you can view the execution records of instances from different perspectives.
- View and manage auto triggered instances from the instance perspective: You can view the execution records of a single instance.
- View and manage auto triggered instances from the workflow perspective: You can view the execution records of all instances in a workflow.

View and manage auto triggered instances from the instance perspective

On the Auto Triggered Instances page, click Instance Perspective to go to the Instance Perspective tab.

周期实例

Item	Description
Filter instances that meet specific conditions (Area 1)	Allows you to specify conditions to search for auto triggered instances in the area marked with 1 in the preceding figure. Note By default, the data timestamp is set to the day before the current day. You can search for your desired instance by instance ID. You can determine whether to use a task ID or an instance ID to search for your desired instance based on your business requirements. If you want to search for all the auto triggered instances that are generated on the current day by an auto triggered task scheduled by hour or minute, you can enter the ID of the task. If you want to search for a specific auto triggered instance that is generated on the current day by an auto triggered task scheduled by hour or minute, you can enter the ID of the auto triggered instance. For instances for which monitoring rules are configured, you can select the Nodes for Which Alerts are Reported Within Last 24 Hours check box to search for the instances for which alerts are triggered within 24 hours from the current point in time. You can click the Alert icon in the DAG of an instance for which alerts are reported within last 24 hours from the current point in time to view the alert details about the instance. For more information, see Overview. You can select the Slow Nodes check box to filter slow running instances. An auto triggered instance is considered a slow running instance if the auto triggered instance is running and the running duration of the auto triggered instance is at least 15 minutes longer than the average running duration of the auto triggered instance during the last 10 days. If the number of historical auto triggered instances is less than four and the running duration of an auto triggered instance exceeds 30 minutes, the auto triggered instance can also be considered a slow running instance.
View the list of instances (Area 2)	Allows you to view the auto triggered instances that meet the specified conditions in the area marked with 2 in the preceding figure. You can view the status, basic information, and runtime of the instances.
Manage instances (Area 3 and Area 4)	Allows you to perform operations on an auto triggered instance in the areas marked with 3 and 4 in the preceding figure. DAG: You can perform this operation to view the dependencies of an auto triggered instance in the DAG of the instance. For more information, see Appendix: Use the features provided in a DAG. Perform Diagnostics: You can perform this operation to carry out end-to-end analysis on an auto triggered instance. If the auto triggered instance is not run as expected, you can click Perform Diagnostics to troubleshoot issues. For more information, see Intelligent diagnosis. Rerun: You can perform this operation to rerun an auto triggered instance that is in the Successful or Failed state. After the auto triggered instance is successfully run, its descendant instances that are in the Not Run state can be scheduled to run. This operation is used to process an auto triggered instance that fails to be run or an auto triggered instance that is not run as scheduled. More: Rerun Descendant Nodes: You can perform this operation to rerun the descendant instances of an auto triggered instance that is in the Successful or Failed state. You can select the descendant instance that you want to rerun. After the selected instance is successfully run, its descendant instances that are in the Not Run state can be scheduled to run. This operation is used to recover data. Set Status to Successful: You can perform this operation to set the status of an auto triggered instance that fails to be run to Successful. You can perform this operation if you do not want an auto triggered instance that fails to be run to block the running of its descendant instances. This operation is used to process an auto triggered instance that fails to be run. Stop: You can perform this operation to stop an auto triggered instance that does not need to be run. After you perform this operation on an auto triggered instance, the auto triggered instance fails to be run and exits. Only auto triggered instances in the Pending (Schedule), Pending (Resources), or Running state can be stopped. Freeze: You can perform this operation if you do not need to run an auto triggered instance and its descendant instances. The freeze operation takes effect only on the current auto triggered instance that is in the Running state. A frozen auto triggered instance cannot be scheduled as expected and does not generate data. After an auto triggered instance is frozen, its descendant instances cannot be scheduled and run as expected. Note Do not perform this operation on the projectname_root task, which is the root task of your workspace. All the instances of auto triggered tasks depend on this task. If this task is frozen, the instances of auto triggered tasks cannot be run. You cannot freeze an auto triggered that is in one of the following states: Pending (Resources), Pending (Schedule), and Running. If the code of the task is being executed or data quality of the task is being checked, the status of the task can be considered running. Unfreeze: You can perform this operation to unfreeze an auto triggered instance that is frozen. If the auto triggered instance is not run, it is automatically run after its ancestor instances are successfully run. If all the ancestor instances of the auto triggered instance are successfully run, the state of the auto triggered instance is directly set to Failed. You must manually rerun the auto triggered instance. Note The unfreeze operation takes effect only on the current auto triggered instance. If the auto triggered for which the instance is generated is frozen, instances that are scheduled to run on the next day are also frozen. View Lineage: You can perform this operation to view the lineage of an auto triggered instance. View Node Details: You can perform this operation to view the basic information about an auto triggered instance. View Runtime Log: After the auto triggered instance for which the test instance is generated is started, you can perform this operation to view the execution details of the auto triggered instance based on the operation logs. For more information about the core parameters in the logs, see Appendix 2: Parameters in operation logs. Change Resource Group: You can perform this operation to change the resource group for scheduling that is used to run an auto triggered instance. This operation does not cause the change to the resource group for scheduling that is used to run the auto triggered for which the auto triggered instance is generated. Batch operations: If you want to perform operations on a large number of instances, you can select the instances in the instance list and perform operations on the instances at a time in the lower part of the Instance Perspective tab. For example, you can stop or rerun the instances at a time.

Manage auto triggered instances in a DAG

周期实例

Note

In a DAG, same-cycle scheduling dependencies are presented as solid lines, and cross-cycle scheduling dependencies are presented as dashed lines. For more information, see Scheduling dependencies.

Functionality	Description
Operations that you can perform on the DAG page	You can click DAG in the Actions column of an auto triggered instance to open the DAG of the instance. You can perform operations such as task aggregation, upstream and downstream analysis, and the display pattern adjustment of the DAG in the areas marked with 1, 2, and 3 in the preceding figure. For more information, see Appendix: Use the features provided in a DAG.
Operations on a single instance	You can right-click your desired auto triggered instance in a DAG and perform operations on the instance. Show Ancestor Nodes: You can perform this operation to view ancestor instances of the current auto triggered instance. You can select this option to have a command of the instances that affect data output of the current instance. You can view ancestor instances of an auto triggered instance by level. A maximum of six levels of ancestor instances can be displayed at the same time. Show Descendant Nodes: You can perform this operation to view descendant instances of the current auto triggered instance. You can select this option to have a command of the instances whose data output is affected by the current instance. You can view descendant instances of an auto triggered instance by level. A maximum of six levels of descendant instances can be displayed at the same time. View Runtime Log: After the auto triggered instance for which the test instance is generated is started, you can perform this operation to view the execution details of the auto triggered instance based on the operation logs. For more information about the core parameters in the logs, see Appendix 2: Parameters in operation logs. View Code: You can perform this operation to view the code of the task for which the current auto triggered instance is generated in the production environment. If the code of the node does not meet your expectations, you must check whether the latest code of the node is successfully deployed to the production environment. Edit Node: You can perform this operation to go to the configuration tab of the task for which the current auto triggered instance is generated on the DataStudio page. View Lineage: You can perform this operation to view the lineage of an auto triggered instance. More: You can view the basic properties, operations logs, and task code of an instance. View Auto Triggered Nodes: You can view the information of the auto triggered task to which the current instance belongs. Go to Task 360 Page: You can go to the Data Governance Center page to view the task running details from multiple perspectives, such as the baseline list and instance running status. Then, you can manage tasks. For more information, see Obtain a panoramic view of a task. Stop: You can perform this operation to stop an auto triggered instance that does not need to be run. After you perform this operation on an auto triggered instance, the auto triggered instance fails to be run and exits. Only auto triggered instances in the Pending (Schedule), Pending (Resources), or Running state can be stopped. Rerun: You can perform this operation to rerun an auto triggered instance that is in the Successful or Failed state. After the auto triggered instance is successfully run, its descendant instances that are in the Not Run state can be scheduled to run. This operation is used to process an auto triggered instance that fails to be run or an auto triggered instance that is not run as scheduled. Rerun Descendant Nodes: You can perform this operation to rerun the descendant instances of an auto triggered instance that is in the Successful or Failed state. You can select the descendant instance that you want to rerun. After the selected instance is successfully run, its descendant instances that are in the Not Run state can be scheduled to run. This operation is used to recover data. Set Status to Successful: You can perform this operation to set the status of an auto triggered instance that fails to be run to Successful. You can perform this operation if you do not want an auto triggered instance that fails to be run to block the running of its descendant instances. This operation is used to process an auto triggered instance that fails to be run. Resume: You can perform this operation to allow the current auto triggered instance to resume running from the position where it is stopped. For example, if an instance is run by executing multiple SQL statement segments, the instance resumes running from the SQL statement segment in which the SQL statements fail to be executed. Note Only SQL tasks that are run based on a MaxCompute compute engine support this operation. If you want to upgrade an exclusive resource group for scheduling that you purchased after January 2021, you can refer to the following instructions: You can click the link for application to join the DataWorks DingTalk group for pre-sales or after-sales services. After you join the DingTalk group, you can directly contact the DingTalk chatbot or contact on-duty technical personnel. The following figure shows the QR code of the DataWorks DingTalk group. Trigger Data Quality Check: If auto triggered tasks are configured with data quality monitoring rules, the system checks the monitoring rule. Emergency Operations: You can perform emergency operations on the current auto triggered instance. The emergency operations take effect only on the current instance once. Delete Dependencies: You can perform this operation to urgently delete dependencies for the current auto triggered instance. In most cases, you can delete dependencies for an auto triggered instance by clicking Delete Dependencies if the ancestor instances of the instance fail to be run and the ancestor instances do not affect data output of the instance. Note You must check whether this operation affects data output based on the code of the task for which the instance is generated and lineage of the instance. Change Priority: You can perform this operation to change the priority of the auto triggered for which the current auto triggered instance is generated based on your business requirements. A larger value indicates a higher priority. The priority of a task depends on the priority of the baseline with which the task is associated. Force Rerun: You can perform this operation to forcefully rerun the current auto triggered instance. You can perform this operation on an auto triggered instance that is in the Successful, Failed, or Not Run state. This operation is often performed to recover data. Force Heavy Run Downstream: You can perform this operation to forcefully rerun the descendant instances of an auto triggered instance whose data timestamp is the previous day or the day before the previous day. You can perform this operation on an auto triggered instance that is in the Successful or Failed state. This operation is often performed to recover data. For more information, see Appendix: Forcefully rerun the descendant instances of an auto triggered node instance. Note You can use only a RAM user to which a workspace administrator or tenant administrator role is assigned, or an Alibaba Cloud account to forcefully rerun the descendant instances of an auto triggered instance. Clone Instance: You can click Clone Instance to create a new instance with the same configurations as a host instance that is in the Running state. The new instance is a cloning instance. The new instance is named in the `dw_clone_Node name` format. Note You can clone only instances generated based on ODPS SQL nodes. Each instance can be cloned only once. Execution logic of the host instance and the cloned instance: Both the host instance and the cloned instance are in the Running state. If the host instance succeeds before the cloned instance, the cloned instance will be stopped. If the cloned instance succeeds before the host instance, the host instance will be stopped and enter the Successful state. If a cloned instance exists in a downstream task of the current task, the cloned instance is not rerun when you rerun the descendant instances of the host instance. Freeze: You can perform this operation if you do not need to run an auto triggered instance and its descendant instances. The freeze operation takes effect only on the current auto triggered instance that is in the Running state. A frozen auto triggered instance cannot be scheduled as expected and does not generate data. After an auto triggered instance is frozen, its descendant instances cannot be scheduled and run as expected. Note Do not perform this operation on the projectname_root task, which is the root task of your workspace. All the instances of auto triggered tasks depend on this task. If this task is frozen, the instances of auto triggered tasks cannot be run. You cannot freeze an auto triggered that is in one of the following states: Pending (Resources), Pending (Schedule), and Running. If the code of the task is being executed or data quality of the task is being checked, the status of the task can be considered running. Unfreeze: You can perform this operation to unfreeze an auto triggered instance that is frozen. If the auto triggered instance is not run, it is automatically run after its ancestor instances are successfully run. If all the ancestor instances of the auto triggered instance are successfully run, the state of the auto triggered instance is directly set to Failed. You must manually rerun the auto triggered instance. Note The unfreeze operation takes effect only on the current auto triggered instance. If the auto triggered for which the instance is generated is frozen, instances that are scheduled to run on the next day are also frozen.
View the detailed information about a single instance	In the area marked with 4 in the preceding figure, you can perform the following operations: View Logs: After the auto triggered instance is started, you can perform this operation to view the execution details of the auto triggered instance based on the operation logs. For more information about the core parameters in the logs, see Appendix 2: Parameters in operation logs. Show Details: You can perform this operation to view detailed information about an auto triggered instance on the following tabs: General, Context, Runtime Log, Operation Log, and Code. For more information, see Appendix 1: Details of an auto triggered instance.

View and manage auto triggered instances from the workflow perspective

On the Auto Triggered Instances page, click Workflow Perspective to go to the Workflow Perspective tab.

Note

From the workflow perspective, a DAG displays only ancestor and descendant instances that belong to the current workflow for an auto triggered instance. If the auto triggered task has ancestor and descendant instances that belong to other workspaces or workflows, you can view the dependencies only on the Instance Perspective tab.

View workflows

业务流程视角

Functionality

Description

Status overview of instances in a workflow

The Workflow column displays the status of normal instances (excluding dry-run instances and frozen instances) in the current workflow and the number of instances in each state by using the following icons:

: the number of running instances in the current workflow
: the number of successful instances in the current workflow
: the number of failed instances in the current workflow
: the number of instances that are not in the preceding states in the current workflow

O&M operations on a workflow

You can perform the following operations on a workflow:

DAG: You can perform this operation to view the DAG of the workflow. From the workflow perspective, auto triggered instances that are scheduled by hour or minute in a workflow are displayed in groups by default. The method used to perform operations on an auto triggered instance in a DAG from the workflow perspective is the same as the method used to perform operations on an auto triggered instance in a DAG from the instance perspective. For more information, see Manage auto triggered instances in a DAG.
Rerun: You can perform this operation to rerun all instances or a specified instance in the workflow.
Terminate: You can perform this operation to terminate the instances that are running in the workflow.
Freeze: You can perform this operation to freeze the workflow. Instances in the workflow will not be run after the operation is performed.
Unfreeze: You can perform this operation to unfreeze a workflow that is frozen. The workflow that is unfrozen is in the Failed state by default, and you can rerun the failed workflow.
Set to Successful: You can perform this operation to set the status of the workflow to successful. After the operation is performed, the instances in the workflow are also in the successful state.

Appendix 1: Details of an auto triggered instance

查看实例详情

Tab	Description
General	On this tab, you can view the scheduling properties of an auto triggered instance in the production environment. For more information about the basic parameters, see Configure basic properties. Relationship between a task ID and an instance ID: If you want to search for all the auto triggered instances that are generated on the current day for an auto triggered scheduled by hour or minute, you can enter the ID of the task. If you want to search for a specific auto triggered instance that is generated on the current day for an auto triggered scheduled by hour or minute, you can enter the ID of the auto triggered instance. Instance status interpretation: If the instance is in the Not Run, Pending (Schedule), Pending (Resources), or Freeze state, you can use the intelligent diagnosis feature to quickly troubleshoot issues. Time spent for waiting for resources: If an auto triggered instance is in the Pending (Resources) state for a long period of time, you can use the intelligent diagnosis feature to identify the instances that occupy resources at the time when the current instance is waiting for resources. Then, you can quickly identify the instances on which exceptions occur and troubleshoot issues. Long running duration: If the running duration of an auto triggered instance is much longer than the average running duration over a historical period of time, you can troubleshoot the issue based on task types: Non-synchronization task: You can consult the owner of the compute engine instance on which the auto triggered instance generated for a non-synchronization task is run. Batch synchronization task: The running speed of an auto triggered instance that is generated for a batch synchronization task may be slow in a specific phase or the instance is in the Pending (Resources) state for a long period of time. For more information, see What do I do if a batch synchronization task runs for an extended period of time? Rule-based monitoring: You can view information about the alert rule that is associated with the node for which the current instance is generated. If no alert rule is associated with the node, you can click Create on the right of the Rule Monitoring parameter to quickly create an alert rule to monitor the status of the node. For more information, see Create a custom alert rule. Note You can view only the information about the alert rule that is used to monitor the status of the task. You cannot view the monitoring rule that is used to monitor the data quality of the task. Baseline monitoring: You can view information about the baseline with which the node that generates the current instance is associated. If the node is not associated with baselines, you can click Create on the right of the Baseline monitoring parameter to quickly create a baseline. For more information, see Manage baselines. Tags: You can view the tags that you specified on the Tag Management page. If the current node has issues to be managed, the issues are also displayed as tags. You can view the details in Data Governance Center.
Context	On this tab, you can view all input and output parameters of the task for which an auto triggered instance is generated. For more information, see Configure input and output parameters.
Runtime Log	On this tab, you can view the execution details of the auto triggered task based on the operation logs after the task is started. For more information about the core parameters in the logs, see Appendix 2: Parameters in operation logs.
Operation log	On this tab, you can view the operation records of a node or an instance, including the operation time, operator, and specific operations.
Code	On this tab, you can view the latest code of the task for which the current auto triggered instance is generated in the production environment. If the code of the node does not meet your expectations, you must check whether the latest code of the node is successfully deployed to the production environment. For more information, see Deploy nodes.

Appendix 2: Parameters in operation logs

After the auto triggered task is started, you can view the execution details of the auto triggered task based on the operation logs. The following table describes the core parameters in the operation logs.

Parameter	Description
SKYNET_ONDUTY	The owner of the task.
SKYNET_PARAVALUE	The scheduling parameters.
SKYNET_TASKID	The ID of the instance.
SKYNET_ID	The ID of the task.
SKYNET_NODENAME	The name of the task.
SKYNET_APPNAME	The name of the workspace.
SKYNET_REGION	The ID of the region in which the workspace resides.
SKYNET_CYCTIME	The time when the instance is scheduled to run.

FAQ

For more information, see Overview.