Change resource groups for scheduling for nodes - DataWorks

You can use the resource group orchestration feature to change resource groups for scheduling for multiple nodes in a workflow at the same time. If multiple resource groups for scheduling exist in your workspace, you can reallocate resource groups for scheduling for nodes in the workspace based on your business requirements. This helps you optimize resource allocation and utilization. This topic describes how to use the resource group orchestration feature of DataWorks.

Prerequisites

A workflow is created. For more information about how to create a workflow, see Create an auto triggered workflow.
One or more resource groups for scheduling are created for your workspace. This way, you can use the resource group orchestration feature to allocate resource groups for scheduling in the current workspace. You can purchase and use serverless resource groups. For more information, see Create and use a serverless resource group.

Background information

The resource group orchestration feature is used to change the resource groups for scheduling for multiple nodes in a specific workflow at the same time during data development. The maximum number of nodes that can be run in parallel on a resource group for scheduling varies based on the specifications of the resource group. You can run nodes on resource groups for scheduling with different specifications based on your business requirements. For more information about the specifications of resource groups for scheduling and the maximum number of parallel nodes that are supported by the resource groups for scheduling, see Billing of serverless resource groups.

Note

For more information about how to change the resource groups for scheduling for multiple nodes in the production environment, see View the auto triggered node list.

Limits

DataWorks allows you to change the resource groups for scheduling only for auto triggered nodes.
Zero load nodes do not occupy scheduling resources. Therefore, you cannot change the resource groups for scheduling for zero load nodes.
Only DataWorks Standard Edition and a more advanced edition support the resource group orchestration feature.

Procedure

Go to the DataStudio page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose Data Development and Governance > Data Development. On the page that appears, select the desired workspace from the drop-down list and click Go to Data Development.
Go to the Resource Group Orchestration tab.
On the DataStudio page, move the pointer over the desired workflow in the Business Flow section and click the icon to the right of the workflow name to go to the Resource Group Orchestration tab.
Note
Before you go to the Resource Group Orchestration tab, you must find the workflow in which you want to change the resource groups for scheduling for nodes.
Change the resource groups for scheduling for multiple nodes.
Note
- You can specify the same resource group for scheduling for multiple nodes in one batch.
- If you want to specify different resource groups for scheduling for multiple nodes, you must perform the operations in batches.
1. Optional: Search for the desired nodes by using specific filter conditions, such as the node name, node type, and engine type.
2. Check the nodes in the search results and select the nodes for which you want to change the resource groups for scheduling.
3. Click Switch Resource Groups and select a resource group for scheduling from the list.
4. Click OK.
You can also perform the following operations on resource groups for scheduling:
- Modify the configurations of a resource group: You can modify the configurations of a serverless resource group or old-version exclusive resource group for scheduling. We recommend that you run tasks on serverless resource groups. After you click Settings to the right of the name of a resource group for scheduling, the Overview page of DataWorks appears. In the left-side navigation pane, you can click Resource Groups. On the Exclusive Resource Groups tab, you can find the desired resource group and click Details in the Actions column to view the details of the exclusive resource group for scheduling and modify the configurations of the exclusive resource group for scheduling based on your business requirements.
- Purchase resource groups for scheduling: If no resource groups for scheduling are available in the current workspace, you can click Purchase Resource Group in the upper-right corner of the Resource Group Orchestration tab.
- Change the resource group for scheduling for a single node: Click Switch in the Actions column to change the resource group for scheduling for the node.
Click the icon to commit the changes to the resource groups.
Deploy the nodes whose resource groups for scheduling are changed.
After the resource groups for scheduling of nodes are changed, you must go to the Deploy page to redeploy the nodes to the production environment. After you redeploy the nodes, the nodes are run on the new resource group for scheduling in the production environment. For more information about how to deploy a node, see Deploy nodes.