This topic describes the variables that are involved in task orchestration.
Overview
Variables are dynamic values in a task flow, including system variables, time variables, and non-time variables. Variables can be dynamically referenced in the format of ${varName} in nodes.
System variables
System variables are variables predefined in the system, such as the ID of a task flow and the point in time when the task flow is run.
Variables except for system variables are collectively referred to as custom variables. The name of a custom variable must be unique. The priority of custom variables is higher than that of system variables. If a custom variable shares the same name with a system variable, the value of the custom variable overwrites that of the system variable.
Time variables
Task flow variables (global variables): Task flow variables are offset based on the data timestamp and can be used on all nodes of the current task flow. By default, the data timestamp is the previous day of the day when the task flow is run.
NoteYou can configure task flow variables on one of the nodes in a task flow. Then, the configurations of the task flow variables are automatically synchronized to other nodes in the task flow.
Node variables: Node variables are offset based on the built-in time variable bizdate(T-1). Node variables can be used only on the current node.
For example, if the current date is August 22, 2023 and the offset of a node variable is set to +1 Day, the node variable indicates August 23, 2023.
Non-time variables
Task flow constants (global variables): Task flow constants are fixed values and can be used on all nodes of the current task flow.
Input variables: Input variables are automatically obtained by Data Management (DMS). You can use ${var_name} to reference an input variable in SQL statements for the current node. You can also use an input variable as a filter condition for a Conditional Branch node. The following variables are considered as input variables:
Upstream variables: Upstream variables are output variables from upstream nodes.
Status variables: For more information, see the Status variables section of this topic.
System variables: For more information, see the System variables section of this topic.
Output variables: Output variables are defined and have their values assigned on the current node. Output variables can be accessed and referenced by downstream nodes.
For example, if you configure an output variable on a Script node, the output variable can be referenced in the SQL statements of a downstream node.
Output variables assigned on Single Instance SQL nodes: You can use a Single Instance SQL node to define the result set of an SQL statement. The result set can be used as a 2D matrix. During the definition process of variables, you can take out an element from the matrix or perform character-based concatenation by using a row vector i and a column vector j.
Script output variables: If the last line of the script file is a JSON string in the format of
{ key1: value1, key2: value2, … }for a Script node, the script task parses the key-value pairs in the JSON string to obtain output variables. For each variable, the name is a key and the value is the value of the key. You can reference a script output variable in the format of${key}in the SQL statements of a downstream node.For example, if the last line of the script file is
echo {"hello": "world"}, the script task parses the key-value pair to obtain an output variable whose name is hello and value is world.
Configure time variables
Parameter | Description |
Variable Name | The name of the custom time variable. Note To delete a configured variable, click the |
Variable Rule | The time format and time offset configurations of the custom time variable.
Note After you configure a time variable, you can reference the variable in the |
icon to the right of the Variable Name field. Time formats
The following table describes the time variables that can be used to define the time format of a custom variable.
Time variable | Description | Sample format | Sample value |
Anno Domini (AD) | G indicates AD. | Gyyyy | AD 2021 |
Year |
| yyyy | 2021 |
Month | M: the month of the year. Valid values of M: [1,12]. Valid values of MM: [01,12] | MM | 08 |
Week |
| ww | 13 |
Day |
| D | 360 |
Day of the week |
| e | 1 |
Ante meridiem (AM) or post meridiem (PM) | a: indicates whether the point in time is before or after the midday. If the point in time is in the range of 00:00 to 11:59, the return value is AM. If the point in time is in the range of 12:00 to 23:59, the return value is PM. | a | AM |
Hour |
| HH | 10 |
Minute | m: the minute of the hour. Valid values of m: [0,59]. Valid values of mm: [00,59]. | m | 27 |
Second |
| ss | 08 |
Time zone | z: the time zone. | z | UTC+08:00 |
The following table describes the sample time formats that use multiple time variables.
Sample format | Sample value |
| 2021-08-12 |
| 20210801 |
| 11:05:21 |
| 20210812 11:05:21 |
System variables
Variable | Description | Example |
sys.flow.start.timestamp | The timestamp generated when the task flow is run. | 2021-05-24T11:20:07.562+08:00 |
sys.flow.start.year | The year when the task flow is run. | 2021 |
sys.flow.start.month | The month of the year when the task flow is run. | 5 |
sys.flow.start.day | The day of the month when the task flow is run. | 24 |
sys.flow.start.hour | The hour of the day when the task flow is run. | 11 |
sys.flow.start.minute | The minute of the hour when the task flow is run. | 20 |
sys.flow.start.second | The second of the minute when the task flow is run. | 7 |
sys.flow.start.milliseconds | The millisecond of the second when the task flow is run. | 562 |
sys.flow.start.timezone | The time zone. | Asia/Shanghai |
sys.flow.biztime | The data timestamp. By default, the data timestamp is the previous day of the day when the task flow is run. | 1621740007562 |
sys.flow.name | The name of the task flow. | dwd_activityDailyPV |
sys.node.name | The name of the node in a task flow. | Single Instance SQL-1 |
Status variables
Variable | Description |
all_success | All of the tasks are run. |
all_failed | All of the tasks fail to be run. |
one_success | A task of the current task flow is run. |
one_failed | A task of the current task flow fails to be run. |
You can use status variables on a Conditional Branch node to make a conditional evaluation in a task flow. The subsequent tasks in the task flow can be run only if the Conditional Branch node meets the specified conditions.