All Products
Search
Document Center

DataWorks:Exclusive resource groups

Last Updated:Dec 03, 2024

This topic provides answers to some frequently asked questions about exclusive resource groups of DataWorks.

In which scenario do I need to use an exclusive resource group for scheduling?

If a task that is not used for data integration needs to access a data source that resides in a virtual private cloud (VPC) or a data source that is configured with a whitelist, you must use an exclusive resource group for scheduling. For more information, see Create and use an exclusive resource group for scheduling.

In which scenario do I need to use an exclusive resource group for Data Integration?

If you want to synchronize data from a data source that resides in a VPC, you must use an exclusive resource group for Data Integration. For more information, see Create and use an exclusive resource group for Data Integration.

How do I change the resource group for scheduling that is used to run a task?

  1. If you want to change the resource group for scheduling used to test the running of a task that is in the development environment, perform the following operation:

    Click Run with Parameters in the top toolbar on the configuration tab of the task. In the Parameters dialog box, select the resource group that you want to use.

    高级运行

  2. If you want to change the resource group for scheduling used to run a task that is in the production environment, use one of the following methods:

    • Change the resource group for scheduling on the configuration tab of the task. Then, commit and deploy the task for the change to take effect.

      On the configuration tab of the task, click the Properties tab in the right-side navigation pane. In the Resource Group section of the Properties tab, select the resource group that you want to use. Then, save the change, and commit and deploy the task for the change to take effect. For more information, see Configure the resource property.资源组

    • Change the resource group for scheduling in Operation Center of the production environment.

      On the DataStudio page, click the 图标 icon in the upper-left corner and choose All Products > Data Development And Task Operation > Operation Center to go to the Operation Center page of the production environment. In the left-side navigation pane of the Operation Center page, choose Auto Triggered Node O&M > Auto Triggered Nodes. On the Auto Triggered Nodes page, find the auto triggered task and choose More > Modify Scheduling Resource Group in the Actions column.

      修改资源组

      Note

      Zero load nodes do not occupy resources. Therefore, you do not need to or cannot change the resource group for scheduling for a zero load node.

How do I change the resource group for Data Integration that is used to run a Data Integration task?

  1. If you want to change the resource group for Data Integration used to test the running of a Data Integration task that is in the development environment, perform the following operation:

    On the configuration tab of the task, click Resource Group configuration in the right-side navigation pane. On the Resource Group configuration tab, select the exclusive resource group for Data Integration that you want to use from the Exclusive Resource Group drop-down list.独享数据集成资源组

    Note

    If this exclusive resource group is also required in the production environment, you must select this resource group for this task and commit and deploy the task to the production environment.

  2. If you want to change the resource group for Data Integration used to run a Data Integration task that is in the production environment, perform the following operations:

    On the DataStudio page, click the 图标 icon in the upper-left corner and choose All Products > Data Development And Task Operation > Operation Center to go to the Operation Center page of the production environment. In the left-side navigation pane of the Operation Center page, choose Auto Triggered Node O&M > Auto Triggered Nodes. On the Auto Triggered Nodes page, find the auto triggered task and choose More > Modify Data Integration Resource Group in the Actions column.

How do I renew a resource group?

Log on to the DataWorks console. Click Resource Group in the left-side navigation pane. On the Resource Groups page, find the resource group that you want to renew and click Renew in the Actions column.

Note
  • If you want to use a RAM user to renew a resource group, you must grant the required permissions to the RAM user. For more information, see User permission management.

  • If you want to switch the renewal method of the resource group between manual renewal and auto renewal, you can perform the operation on the Renewal Management page based on the instance name of the resource group.

How do I change the specifications of an exclusive resource group? How do I scale in or out an exclusive resource group?

Log on to the DataWorks console. Click Resource Group in the left-side navigation pane. The Exclusive Resource Groups tab appears. On the Exclusive Resource Groups tab, find the desired exclusive resource group and click Scale Out, Scale In, or Change Specifications in the Actions column to perform the related operation.

Note
  • If you want to use a RAM user to renew a resource group, you must grant the required permissions to the RAM user. For more information, see User permission management.

  • If you scale in or out a resource group, the number of resources in the resource group is decreased or increased.

What conditions must be met before an exclusive resource group can access a data source that resides in a VPC?

If you want to use an exclusive resource group to access a data source that resides in a VPC in DataWorks, the following conditions must be met:

  • The exclusive resource group resides in the same zone as the data source and is associated with a VPC.

  • The exclusive resource group is associated with the same VPC and vSwitch as the data source.

  • If the data source is configured with a whitelist or security group, the elastic IP address (EIP) of the exclusive resource group or the Classless Inter-Domain Routing (CIDR) block of the VPC or vSwitch with which the exclusive resource group is associated is added to the whitelist or security group. For more information, see Configure an IP address whitelist and Appendix: Configure a security group for an ECS instance on which a self-managed database is hosted.

How do I view the network environment of a data source?

  • If you want to view the network environment of an ApsaraDB RDS data source, perform the following steps:

    1. Log on to the ApsaraDB RDS console.

    2. In the left-side navigation pane, click Instances. On the Instances page, find the instance whose network environment you want to view and click the instance name to go to the details page of the instance.

    3. In the left-side navigation pane of the instance details page, click Database Connection.

    4. On the Database Connection page, click Switch VSwitch.

    5. In the Switch VSwitch dialog box, view the vSwitch with which the data source is associated.

  • If you want to view the network environment of a self-managed data source hosted on an Elastic Compute Service (ECS) instance, perform the following steps:

    1. Log on to the ECS console.

    2. In the left-side navigation pane, choose Instances & Images > Instances. On the Instances page, find the instance whose network environment you want to view and click the instance name to go to the details page of the instance.

    3. On the instance details page, view the configuration information of the instance.

What information about an exclusive resource group must be added to the whitelist of a data source that the exclusive resource group needs to access?

  • If you want to access a data source over a VPC, you must add the CIDR block of the vSwitch with which the resource group is associated to the IP address whitelist of the data source.

    On the Exclusive Resource Groups tab of the Resource Groups page in the DataWorks console, find the desired resource group and click Network Settings in the Actions column. On the VPC Binding tab of the page that appears, view and record the CIDR block of the related vSwitch. Then, add the CIDR block to the IP address whitelist of the data source.

  • If you want to access a data source over the Internet, you must perform one of the following operations to configure the IP address whitelist of the data source:

    • If you use a serverless resource group, you must add the EIP configured for the VPC with which the resource group is associated to the IP address whitelist of the data source.

      On the Internet NAT Gateway page of the VPC console, find the source network address translation (SNAT) entry that is configured, and obtain the public IP address that is associated with the related vSwitch. Then, add the public IP address to the IP address whitelist of the data source.

      image

    • If you use an old-version resource group, you must add the EIP of the resource group to the IP address whitelist of the data source.

      On the Exclusive Resource Groups tab of the Resource Groups page in the DataWorks console, find the desired resource group and click Details in the Actions column. In the Basic Information section of the page that appears, view and record the EIP displayed next to the EIPAddress parameter. Then, add the EIP to the IP address whitelist of the data source.

      Note

      If you scale out the resource group in subsequent operations, you must check whether the EIP changes. If the EIP changes, we recommend that you add the latest EIP to the IP address whitelist of the data source at the earliest opportunity after the scale-out operation. This ensures that your task can run as expected.

How do I associate an exclusive resource group with or disassociate an exclusive resource group from a workspace?

Log on to the DataWorks console. Click Resource Group in the left-side navigation pane. The Exclusive Resource Groups tab appears. On the Exclusive Resource Groups tab, find the exclusive resource group for which you want to change the workspace and click Change Workspace in the Actions column. In the Modify home workspace dialog box, you can associate the resource group with or disassociate the resource group from a workspace.

Note

You must obtain the administrator permissions of the workspace before you can perform the preceding operations.

If the resources in exclusive resource groups on which a data synchronization task runs are insufficient, how do I scale out the exclusive resource groups?

You need to only scale out the exclusive resource group for Data Integration for the data synchronization task.