All Products
Search
Document Center

Simple Log Service:Authorize a default role to write data to MaxCompute

Last Updated:Jan 06, 2025

When a data shipping job of the new version is running to ship data to MaxCompute, the data that is read from a Logstore must be written to a MaxCompute table. To meet the requirement, you can authorize the data shipping job to assume a default role.

Prerequisites

  • If you use a Resource Access Management (RAM) user, make sure that the RAM user has the permissions to manage RAM roles.

  • A MaxCompute project is added to the required DataWorks workspace as the data source. For more information, see Add a MaxCompute data source.

Ship data within an Alibaba Cloud account

If Simple Log Service and MaxCompute belong to the same Alibaba Cloud account, you must complete authorization on the Cloud Resource Access Authorization page and create a role named AliyunLogDefaultRole. For more information, see Cloud Resource Access Authorization. After the AliyunLogDefaultRole role is created, you must add the role as a workspace member.

Procedure

  1. Modify the trust policy of the RAM role.

    1. Log on to the RAM console.

    2. In the left-side navigation pane, choose Identities > Roles.

    3. In the role list, find the AliyunLogDefaultRole role and click the role name.

    4. On the Trust Policy tab, click Edit Trust Policy.

    5. Replace the existing script in the code editor with the following policy document and click Save trust policy document:

      {
        "Statement": [
          {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Principal": {
              "Service": [
                "log.aliyuncs.com",
                "dataworks.aliyuncs.com"
              ]
            }
          }
        ],
        "Version": "1"
      }
  2. Add the RAM role as a workspace member.

    You can use the GUI or CLI to grant permissions to the RAM role.

    Use the GUI

    1. Log on to the DataWorks console.

    2. In the top navigation bar, select a region.

    3. In the left-side navigation pane, click Workspace.

    4. On the Workspaces page, find the workspace that you want to manage and click Manage in the Actions column.

    5. In the Workspace Settings panel, click Workspace Members and Roles. On the Workspace Members tab of the Workspace page, click Add Members.image

    6. In the Add Members dialog box, select the current logon account and the AliyunLogDefaultRole role, and then add the logon account and the AliyunLogDefaultRole role as prompted.

      In the Batch Assign Roles section, select Development. For more information, see Grant permissions to a RAM user.image

    7. Authorize AliyunLogDefaultRole to manage a MaxCompute table.

      1. Log on to the MaxCompute console. In the top navigation bar, select a region.

      2. In the left-side navigation pane, choose Workspace > Projects. On the Projects page, find the project that you want to manage and click Manage in the Actions column.

      3. On the MaxCompute project management page, click the Role Permissions tab.image.png

        If the following error is reported, perform the following operations in the role list: Find the admin role and click Manage Members in the Actions column. In the Manage Members dialog box, select the current logon account and add the logon account as prompted.image.png

      4. In the role list, find the role_project_admin role and click Manage Members in the Actions column.

      5. In the Manage Members dialog box, select the current logon account and the AliyunLogDefaultRole, and then add the logon account and the AliyunLogDefaultRole role as prompted.

      6. In the role list, find the role_project_admin role and click Edit Role in the Actions column.

      7. On the Table tab of the Edit Role dialog box, find the MaxCompute table that you want to manage and select Describe, Alter, and Update.image.png

      Important

      The preceding authorization procedure takes effect only on the specified MaxCompute table. If you want to authorize the AliyunLogDefaultRole role to manage all tables in the current MaxCompute project, you can grant the permissions of the admin role to the current logon account and the AliyunLogDefaultRole role. In the role list, find the admin role and click Manage Members in the Actions column. In the Manage Members dialog box, select the current logon account and the AliyunLogDefaultRole role, and then add the logon account and the AliyunLogDefaultRole role as prompted.

    Use the CLI

    1. Log on to the DataWorks console.

    2. In the top navigation bar, select a region.

    3. In the left-side navigation pane, click Workspace.

    4. On the Workspaces page, find the workspace that you want to manage, move the pointer over Shortcuts, and then click Data Development.

    5. Create a workflow.

      1. On the Scheduled Workflow page, choose Create > Create Workflow.

      2. In the Create Workflow dialog box, configure the Workflow Name parameter and click Create.

    6. Create a node.

      1. On the Scheduled Workflow page, choose Create > Create Node > MaxCompute > ODPS SQL.

      2. In the Create Node dialog box, configure the Name and Path parameters, and click Confirm.

        You must set the Path parameter to the workflow that you created in the previous step.

    7. In the code editor of the node, run the required commands to complete the authorization. The following table describes the commands.

      MaxCompute授权

      Command

      Description

      USE project-name;

      Specifies a MaxCompute project. The MaxCompute project must be the same as the MaxCompute project that you specified when you created your data shipping job of the new version. For more information, see Create a data shipping job of the new version to ship data to MaxCompute.

      ADD USER RAM$****.aliyunid.com:`role/aliyunlogdefaultrole`;

      Adds the RAM role as a user to the MaxCompute project.

      • ****.aliyunid.com specifies the Alibaba Cloud account to which the MaxCompute project belongs. You can run the list users; command to view the Alibaba Cloud account.

      • aliyunlogdefaultrole specifies the name of the AliyunLogDefaultRole role. The name must be in lowercase.

      GRANT CreateInstance ON PROJECT project-name TO USER RAM$*****:`role/aliyunlogdefaultrole`;

      Authorizes the RAM role to call the CreateInstance operation to create an instance in a MaxCompute project named project-name.

      project-name specifies the name of the MaxCompute project.

      GRANT DESCRIBE, ALTER, UPDATE ON TABLE table-name TO user RAM$****.aliyunid.com:`role/aliyunlogdefaultrole`;

      Authorizes the RAM role to view, modify, and update the specified MaxCompute table.

      table-name specifies the name of the MaxCompute table.

      Note

      The authorization procedure takes effect only on the specified MaxCompute table. If you want to authorize the RAM role to manage all tables in the current MaxCompute project, run the GRANT admin to user RAM$****.aliyunid.com:`role/aliyunlogdefaultrole`; command to complete the authorization.

      SHOW GRANTS FOR `RAM$****.aliyunid.com:role/aliyunlogdefaultrole`;

      Checks whether the authorization is successful.

      If information similar to the following code is returned, the authorization is successful:

      Authorization Type: ACL
      [user/RAM$****.aliyunid.com:role/aliyunlogdefaultrole]
      A       projects/default_project_****: CreateInstance
      A       projects/default_project_****/tables/****: Describe | Alter | Update

      The following error messages may appear during the authorization process:

What to do next

After you configure the settings, you can assign the AliyunLogDefaultRole role to a MaxCompute data shipping job of the new version to ship data to the specified MaxCompute table. When you create the data shipping job, set the Write Permissions on MaxCompute parameter to Default Role. For more information, see Create a data shipping job of the new version to ship data to MaxCompute.MaxCompute投递授权

Ship data across Alibaba Cloud accounts

If Simple Log Service is activated for Alibaba Cloud Account A and MaxCompute is activated for Alibaba Cloud Account B, you must complete authorization on the Cloud Resource Access Authorization page and create the AliyunLogDefaultRole role in each account. For more information, see Cloud Resource Access Authorization. Then, perform the following operations:

  1. Modify the trust policy of the AliyunLogDefaultRole role that belongs to Alibaba Cloud Account B.

    1. Log on to the RAM console by using Alibaba Cloud Account B.

    2. In the left-side navigation pane, choose Identities > Roles.

    3. In the role list, find the AliyunLogDefaultRole role and click the role name.

    4. On the Trust Policy tab, click Edit Trust Policy.

    5. Replace the existing script in the code editor with the following policy document and click Save trust policy document.

      Add ID of Alibaba Cloud Account A@log.aliyuncs.com and dataworks.aliyuncs.com to the Service element. Replace ID of Alibaba Cloud Account A with the actual ID. You can view the ID of your Alibaba Cloud account in the Account Management console.

      The following policy allows Alibaba Cloud Account A to obtain a temporary Security Token Service (STS) token to manage the cloud resources of Alibaba Cloud Account B:

      {
        "Statement": [
         {
           "Action": "sts:AssumeRole",
           "Effect": "Allow",
           "Principal": {
             "Service": [
               "ID of Alibaba Cloud Account A@log.aliyuncs.com",
               "dataworks.aliyuncs.com",
               "log.aliyuncs.com"
             ]
           }
         }
        ],
        "Version": "1"
      }
  2. Add the AliyunLogDefaultRole role of Alibaba Cloud Account B as a workspace member.

    You can use the GUI or CLI to grant permissions to the RAM role. To complete the authorization, log on to the DataWorks console by using Alibaba Cloud Account B. For more information, see Use the GUI to grant permissions to the RAM role or Use the CLI to grant permissions to the RAM role.

What to do next

After you configure the settings, you can assign the AliyunLogDefaultRole role of Alibaba Cloud Account B to a MaxCompute data shipping job of the new version to ship data to the specified MaxCompute table. When you create the data shipping job, set the Write Permissions on MaxCompute parameter to Custom Role and enter the Alibaba Cloud Resource Name (ARN) of the AliyunLogDefaultRole role that belongs to Alibaba Cloud Account B. In this example, acs:ram::11**13:role/aliyunlogdefaultrole is used. For more information about how to obtain the ARN of a RAM role, see View the information about a RAM role.MaxCompute授权