When a data shipping job of the new version is running to ship data to MaxCompute, the data that is read from a Logstore must be written to a MaxCompute table. To meet the requirement, you can authorize the data shipping job to assume a default role.
Prerequisites
If you are using a Resource Access Management (RAM) user, make sure that the RAM user has the permissions to manage RAM roles.
Ship data within an Alibaba Cloud account
If Simple Log Service and MaxCompute belong to the same Alibaba Cloud account, you must complete authorization on the Cloud Resource Access Authorization page and create the AliyunLogDefaultRole role. After the AliyunLogDefaultRole role is created, you must add the AliyunLogDefaultRole role as a workspace member.
You can use the GUI or the CLI to grant permissions to the RAM role.
Use the GUI to grant permissions to the RAM role
Modify the trust policy of the RAM role.
Log on to the RAM console.
In the left-side navigation pane, choose
.On the Roles page, find the AliyunLogDefaultRole role and click the role name.
On the Trust Policy tab, click Edit Trust Policy.
Replace the content in the code editor with the following policy document and click Save trust policy document.
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "log.aliyuncs.com", "dataworks.aliyuncs.com" ] } } ], "Version": "1" }
Add the RAM role as a workspace member.
Log on to the DataWorks console.
In the upper-left corner of the page that appears, select a region.
In the left-side navigation pane, click Workspaces.
On the Workspaces page, find the workspace that you want to manage and click Manage in the Actions column.
On the Space members tab of the Work space page, click Add Members.
In the Add Members dialog box, select the current logon account and the AliyunLogDefaultRole role, and then add the logon account and the AliyunLogDefaultRole role as prompted.
In the Batch Setting Roles section, select Development. For more information, see Grant permissions to a RAM user.
Grant the AliyunLogDefaultRole role the permissions to manage a MaxCompute table.
On the Work space page, choose
.On the MaxCompute project management page, click Role Permissions.
If the following error is reported, perform the following operations in the role list: Find the admin role and click Manage Members in the Actions column. In the Manage Members dialog box, select the current logon account and add the logon account as prompted.
In the role list, find the role_project_admin role and click Manage Members in the Actions column.
In the Manage Members dialog box, select the current logon account and the AliyunLogDefaultRole role, and then add the logon account and the AliyunLogDefaultRole role as prompted.
In the role list, find the role_project_admin role and click Edit Role in the Actions column.
On the Table tab of the Edit Role dialog box, find the MaxCompute table that you want to manage and select Describe, Alter, and Update.
ImportantThe preceding authorization takes effect only on the specified MaxCompute table. If you want to grant the AliyunLogDefaultRole role the permissions to manage all tables in the current MaxCompute project, you can grant the permissions of the admin role to the current logon account and the AliyunLogDefaultRole role. In the role list, find the admin role and click Manage Members in the Actions column. In the Manage Members dialog box, select the current logon account and the AliyunLogDefaultRole role, and then add the logon account and the AliyunLogDefaultRole role as prompted.
After you configure the settings, you can assign the AliyunLogDefaultRole role to a MaxCompute data shipping job of the new version to ship data to the specified MaxCompute table. When you create the data shipping job, select Default Role for Write Permissions on MaxCompute. For more information, see Create a data shipping job of the new version to ship data to MaxCompute.
Use the CLI to grant permissions to the RAM role
Log on to the DataWorks console.
In the upper-left corner of the page that appears, select a region.
In the left-side navigation pane, click Workspaces.
On the Workspaces page, find the workspace that you want to manage, move the pointer over Shortcuts, and then click Data Development.
Create a workflow.
On the Scheduled Workflow page, choose
.In the Create Workflow dialog box, configure the Workflow Name parameter and click Create.
Create a node.
On the Scheduled Workflow page, choose
.In the Create Node dialog box, configure the Name and Path parameters, and click Confirm.
You must set the Path parameter to the workflow that you created in Step 5.
In the code editor of the node, run the required commands to complete the authorization. The following table describes the commands.
Command
Description
USE project-name;
Specifies a MaxCompute project. The MaxCompute project must be the same as the MaxCompute project that you specified when you created your data shipping job of the new version. For more information, see Create a data shipping job of the new version to ship data to MaxCompute.
ADD USER RAM$****.aliyunid.com:`role/aliyunlogdefaultrole`;
Adds the RAM role as a user to the MaxCompute project.
****.aliyunid.com specifies the Alibaba Cloud account to which the MaxCompute project belongs. You can run the
list users;
command to view the Alibaba Cloud account.aliyunlogdefaultrole specifies the name of the AliyunLogDefaultRole role. The name must be in lowercase letters.
GRANT CreateInstance ON PROJECT project-name TO USER RAM$*****:`role/aliyunlogdefaultrole`;
Grants the RAM role the permission to create instances in the specified project.
GRANT DESCRIBE, ALTER, UPDATE ON TABLE table-name TO user RAM$****.aliyunid.com:`role/aliyunlogdefaultrole`;
Grants the RAM role the permissions to view, modify, and update the specified MaxCompute table.
NoteThe authorization procedure applies only to the specified MaxCompute table. If you want to grant the RAM role the permissions to manage all tables in the current MaxCompute project, run the
GRANT admin to user RAM$****.aliyunid.com:`role/aliyunlogdefaultrole`;
command to complete the authorization.SHOW GRANTS FOR `RAM$****.aliyunid.com:role/aliyunlogdefaultrole`;
Checks whether the authorization is successful.
If information similar to the following code is returned, the authorization is successful:
Authorization Type: ACL [user/RAM$****.aliyunid.com:role/aliyunlogdefaultrole] A projects/default_project_****: CreateInstance A projects/default_project_****/tables/****: Describe | Alter | Update
The following error messages may appear during the authorization process:
If the
FAILED: mismatched input
error message appears, the RAM user does not have the permissions to execute statements such asADD USER
. For more information, see What do I do if the error message FAILED: mismatched input appears?If the
FAILED: ODPS-0130013:Authorization exception - Authorization Failed [4003], You have NO privilege to do the PROJECT SECURITY OPERATION for {acs:odps:*:projects/xxxxxx/authorization/users}. Context ID:1111-11111-1111-1111-11111.
error message appears, the user does not have the permissions to manage MaxCompute projects or grant permissions to other users. For more information, see What do I do if the system reports an error that a user does not have the required permissions?
After you configure the settings, you can assign the AliyunLogDefaultRole role to a MaxCompute data shipping job of the new version to ship data to the specified MaxCompute table. When you create the data shipping job, select Default Role for Write Permissions on MaxCompute. For more information, see Create a data shipping job of the new version to ship data to MaxCompute.
Ship data across Alibaba Cloud accounts
If Simple Log Service is activated for Alibaba Cloud Account A and MaxCompute is activated for Alibaba Cloud Account B, you must complete authorization on the Cloud Resource Access Authorization page and create the AliyunLogDefaultRole role in each account. Then, perform the following operations:
Modify the trust policy of the AliyunLogDefaultRole role that belongs to Alibaba Cloud Account B.
Log on to the RAM console by using Alibaba Cloud Account B.
In the left-side navigation pane, choose .
In the role list, click AliyunLogDefaultRole.
On the Trust Policy tab, click Edit Trust Policy.
Replace the content in the code editor with the following policy document and click Save trust policy document.
Add ID of Alibaba Cloud Account A@log.aliyuncs.com and dataworks.aliyuncs.com to the Service element. Replace ID of Alibaba Cloud Account A with the actual ID. You can view the ID of Alibaba Cloud account A in the Account Management console.
The following policy allows Alibaba Cloud Account A to obtain a temporary Security Token Service (STS) token to manage the cloud resources of Alibaba Cloud Account B:
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "ID of Alibaba Cloud Account A@log.aliyuncs.com", "dataworks.aliyuncs.com", "log.aliyuncs.com" ] } } ], "Version": "1" }
Add the AliyunLogDefaultRole role that belongs to Alibaba Cloud Account B as a workspace member.
You can use the GUI or the CLI to grant permissions to the RAM role. To complete the authorization, log on to the DataWorks console by using Alibaba Cloud Account B. For more information, see Use the GUI to grant permissions to the RAM role or Use the CLI to grant permissions to the RAM role.
After you configure the settings, you can assign the AliyunLogDefaultRole role of Alibaba Cloud Account B to a MaxCompute data shipping job of the new version to ship data to the specified MaxCompute table. When you create the data shipping job, select Custom Role for Write Permissions on MaxCompute and enter the Alibaba Cloud Resource Name (ARN) of the AliyunLogDefaultRole role that belongs to Alibaba Cloud Account B. In this example, acs:ram::11**13:role/aliyunlogdefaultrole is used. For more information about how to obtain the ARN of a RAM role, see View the information about a RAM role.