MaxCompute:Purchase and use exclusive resource groups for data transmission service

Background information

Check network and region conditions

Purchase exclusive resource groups for data transmission service

1. Log on to the MaxCompute console. In the upper-left corner, select the region, then choose Workspace > Quotas from the left-side navigation pane.

2. On the Quotas page, click New Quota to navigate to the resource purchase page. Follow the prompts to configure the purchase parameters. Key parameters include:

   Parameter name	Description
   Specifications Type	Select Tunnel.
   Tunnel Concurrency	Select the number of concurrency you wish to purchase. Note The minimum purchase is 50 concurrency, with increments in multiples of 1.

3. Click Buy Now. Agree to the service terms and complete the payment process as prompted to finalize the purchase.

   Once the purchase is complete, return to the console. A new quota group with the Quota Type of Data Transmission Service appears on the Quotas page. You can manage this quota group with options to Upgrade, Downgrade, or Renew. For policies on upgrades and downgrades, see Upgrade and downgrade.

Use exclusive resource groups for data transmission service

For data transmission or integration tasks, you can specify exclusive resource groups for data transmission service to enhance data processing efficiency based on business needs.

1. Access the MaxCompute project through the MaxCompute client (odpscmd) or other tools, then enable the exclusive resource group for data transmission service.

    setproject odps.tunnel.enable.quota.route.v2=true;

   Note

   Ensure your MaxCompute SDK is updated to version 0.45 or later.

2. Grant required permissions.

   By default, all Alibaba Cloud accounts, RAM users, and roles do not have permissions to specify a quota group for a job. An account can be used to specify a quota group for a job only after the account is granted the required permissions.

   1. Create a role.

      1. Log on to the MaxCompute console. In the top navigation bar, select a region.

      2. In the left-side navigation pane, choose Tenants > Roles.

      3. On the Roles page, click Add Role.

      4. In the Add Role dialog box, configure the Role Name parameter and enter the following policy document:

         {
             "Statement": [{
                     "Action": [
                         "odps:List",
                         "odps:Usage"],
                     "Effect": "Allow",
                     "Resource": ["acs:odps:*:regions/*/quotas/*"]}],
             "Version": "1"
         }

         The following table describes the parameters in the sample code.

         Parameter	Description
         Action	The action that is allowed or denied. You can specify multiple actions for a single authorization operation based on your business requirements. If you specify multiple actions, separate the actions with commas (,). For more information about the supported actions, see MaxCompute permissions. Note For more information about policy parameters, see Policy elements.
         Resource	The authorized resource range. Configure this parameter in the ["acs:odps:Tenant/${tenant_id}:regions/${region_id}/quotas/${quota_name}"] format. ["acs:odps:*:regions/*/quotas/*"] specifies all quota groups in all regions for the current tenant. You can specify a specific region and a specific quota group based on your business requirements.

      5. Click OK.

   2. Assign the role that you created to the account that you want to use to specify the quota group at the job level.

      Note

      By default, the Alibaba Cloud account or a RAM user that is assigned the tenant-level Super_Administrator role can perform the authorization operations.

      The authorization operations vary based on the type of the object to which permissions are granted.

      • Alibaba Cloud account

        To grant permissions to an Alibaba Cloud account, run the following commands:

        -- Add the Alibaba Cloud account to the current tenant and assign the specified role to the Alibaba Cloud account.
        Add tenant user <Aliyun$xxxx>;
        Grant tenant role <role_name> to user <Aliyun$xxxx>;
        -- View the permissions of the tenant-level role or user.
        Show grants for tenant role <role_name>;
        Show grants for tenant user <user_name>;
        Show principals for tenant [role] <role_name>;

      • RAM user

        1. In the left-side navigation pane of the MaxCompute console, choose Tenants > Users.

        2. On the Users page, find the required RAM user and click Modify Role in the Actions column.

        3. In the Edit Role dialog box, select the required role in the Available Roles section to add the role to the Added Roles section.

        4. Click OK.

3. Manually specify the quota group in the SDK of the data transmission service task.

   --Use the resource group with the quota name XXX for data transmission service.
   final TableTunnel tunnel = new TableTunnel(odps);
   tunnel.getConfig().setQuotaName("xxx");
   TableTunnel.UploadSession up = tunnel.createUploadSession(projectName, tableName);
   
   --You can get the resource group actually used by the session
   up.getQuotaName();

   Log on to the MaxCompute console, choose Workspace > Quotas, view and copy the corresponding Quota Name from the Quotas page. For example, ot_42854300324****_169821756****_p#ot_42854300324****_169821756****.

   

View resource group metrics

Other operations

You can configure the resource group through the MaxCompute console by following these steps:

1. Log on to the MaxCompute console. In the upper-left corner, select a region. In the left-side navigation pane, choose Workspace > Projects.

2. On the Projects page, select the desired project. Click Manage in the Actions column, then Edit in the Basic Properties section.

3. From the Data Transmission Service dropdown, select the resource group you wish to attach.

   You can choose the default public resource group or the purchased exclusive resource group.

   Data Transmission Service resource group	Enable As Default Data Transmission Service	Description
   default	-	Sets the public data transmission service resource group as the default. All data transmission tasks within this project use the public resource group and cannot access the exclusive resource group.
   Exclusive Resource Group	Yes	Sets the exclusive data transmission service resource group as the default. For individual tasks, manually specify the QuotaName to target the exclusive resource group when submitting tasks through the SDK or data integration.
   	No	Sets the public data transmission service resource group as the default. For individual tasks, manually specify the QuotaName for the exclusive resource group when submitting tasks through the SDK or data integration.