DataWorks:Use workflow parameters

Supported node types

The workflow parameter feature is supported only for the following types of nodes: ODPS SQL nodes, EMR Spark Shell nodes, EMR Spark nodes, EMR Shell nodes, EMR Hive nodes, EMR MR nodes, EMR Presto nodes, EMR Trino nodes, EMR Spark SQL nodes, EMR Spark Streaming nodes, EMR Streaming SQL nodes, and Hologres SQL nodes. You can view the node types that support the workflow parameter feature in the DataWorks console.

Precautions

• After you configure a workflow parameter in a workflow and save the configurations of the workflow parameter, each time you open a node that supports the workflow parameter in the workflow, the system loads the latest configuration of the workflow parameter to the parameter list of the node, and the node becomes editable. To allow the latest configuration to take effect in scheduling scenarios, you must commit and deploy the node.

• If the workflow parameter that you configure is the same as a parameter of a node but the value of the workflow parameter is different from the value of the node parameter, the value of the workflow parameter overwrites the value of the node parameter.

• When you configure a workflow parameter, you cannot set the Value/Expression parameter for the workflow parameter in the format of ${English word}:. For example, you cannot specify the parameter value in the format of var:1 or specify the parameter expression in the format of ${var}:1.

Go to the entry point for configuring a workflow parameter

1. Go to the DataStudio page.

   Log on to the DataWorks console. In the left-side navigation pane, choose Data Modeling and Development > DataStudio. On the page that appears, select the desired workspace from the drop-down list and click Go to DataStudio.

2. Go to the entry point for configuring a workflow parameter.

   In the Scheduled Workflow pane, find the desired workflow and double-click its name. In the right-side navigation pane of the configuration tab of the workflow, click Workflow Parameters.

Configure a workflow parameter

For all types of nodes that support the workflow parameter feature, the methods that are used to configure workflow parameters are the same. If you need to assign a value to the same parameter used for multiple nodes in a workflow at the same time, we recommend that you use the workflow parameter feature. In this example, the workflow parameter feature is used to replace the value 20230613 of the ReplaceMe parameter with the value 20230615 for an ODPS SQL node in an auto triggered workflow.

1. Configure node parameters for an ODPS SQL node used for data development.

   1. Go to the configuration tab of an ODPS SQL node.

      Double-click the ODPS SQL node in the desired workflow to go to the configuration tab of the node.

   2. Configure node parameters for the node.

      Enter the ReplaceMe parameter and assign the value 20230613 to the parameter in the Parameters section of the Properties tab, as shown in the following figure. For more information about how to configure node parameters for a node, see Configure scheduling parameters.

2. Replace the value of the ReplaceMe parameter of the ODPS SQL node by using the workflow parameter feature.

   1. In the Scheduled Workflow pane, find the desired workflow and double-click its name. In the right-side navigation pane of the configuration tab of the workflow, click Workflow Parameters.

   2. On the Workflow Parameters tab, configure the Parameter Name and Value/Expression parameters.

      In the following figure, the value of the Value/Expression parameter is set to 20230615. After you save the configuration, the value of the ReplaceMe parameter of the ODPS SQL node in the workflow is replaced with 20230615. You can click Add on the Workflow Parameters tab to add more workflow parameters based on your business requirements.

      Note

      • If the workflow parameter that you configure is the same as a parameter of a node but the value of the workflow parameter is different from the value of the node parameter, the value of the workflow parameter overwrites the value of the node parameter.

      • When you configure a workflow parameter, you cannot set the Value/Expression parameter for the workflow parameter in the format of ${English word}:. For example, you cannot specify the parameter value in the format of var:1 or specify the parameter expression in the format of ${var}:1.

      The following figure shows how the value of a node parameter of the ODPS SQL node is replaced.

Check whether the configurations of the workflow parameter take effect

After you configure the workflow parameter, you must run the workflow to check whether the value specified for the workflow parameter replaces the value of the node parameter of the node.

1. Run the workflow and assign a value to each workflow parameter.

   If workflow parameters are used in the workflow that you want to run, you must manually assign a value to each workflow parameter. For information about more running methods, see Debugging procedure.

   1. In the top toolbar of the configuration tab of the workflow, click the icon.

   2. In the Enter parameters dialog box, assign the value 20230615 to the workflow parameter ReplaceMe.

2. View the running result.

   On the configuration tab of the workflow, right-click the ODPS SQL node and select View Log. Then, you can view the value that is assigned to the ReplaceMe parameter of the ODPS SQL node. The value of the node parameter of the ODPS SQL node is changed from 20230613 to 20230615. This indicates that the workflow parameter takes effect and the workflow runs as expected.

Commit and deploy the workflow parameter

After you confirm that the workflow parameter is correctly configured, you must commit and deploy the current workflow. After the workflow is committed and deployed, the workflow parameter can take effect in scheduling scenarios.

1. Commit the workflow.

   Perform the steps that are shown in the following figure to commit the workflow. In the Commit dialog box that appears after you click Submit, select the node that you want to commit and configure the Change Description parameter.

   Note

   You can specify whether to use the code review feature based on your business requirements. If you want to use the feature, the code of the node that you select can be deployed only if the node code passes the review.

   

2. Deploy the workflow.

   If the workflow is created in a workspace in standard mode, after the workflow is committed, you must click the Deploy icon in the upper-right corner of the configuration tab of the workflow to deploy nodes in the workflow. For more information, see Deploy nodes.