You can assign a custom role to a data transformation job to read data from a source Logstore and write transformed data to a destination Logstore. This topic describes how to grant a custom role the permissions to read data from a Logstore.
Prerequisites
A Resource Access Management (RAM) role is created. For more information, see Create a RAM role for a trusted Alibaba Cloud service. Note that the RAM role must meet the following requirements:
When creating the RAM role, select Alibaba Cloud Service as the trusted entity.
In the trust policy of the RAM role, the
"Service"field must include"log.aliyuncs.com", as illustrated below.{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "log.aliyuncs.com" ] } } ], "Version": "1" }
Grant the RAM role the permissions to read data from a source Logstore
After you use an Alibaba Cloud account to grant permissions to the RAM role, the RAM role can read data from the source Logstore. When you create a data transformation job, you can use the RAM role. For more information, see Create a data transformation job.
Log on to the RAM console by using your Alibaba Cloud account.
Create a policy.
The policy grants the permissions to read data from the source Logstore.
In the left-side navigation pane, choose .
On the Policies page, click Create Policy.
On the Create Policy page, configure the following information and click OK.
Configuration item
Description
Configuration mode
Click JSON.
Policy document
Replace the content in the editor with the following script.
Policy that uses exact match
In this example, the source project name is log-project-prod, and the source Logstore name is access_log. Replace the project and Logstore names based on your business requirements.
{ "Version": "1", "Statement": [ { "Action": [ "log:ListShards", "log:GetCursorOrData", "log:GetConsumerGroupCheckPoint", "log:UpdateConsumerGroup", "log:ConsumerGroupHeartBeat", "log:ConsumerGroupUpdateCheckPoint", "log:ListConsumerGroup", "log:CreateConsumerGroup" ], "Resource": [ "acs:log:*:*:project/log-project-prod/logstore/access_log", "acs:log:*:*:project/log-project-prod/logstore/access_log/*" ], "Effect": "Allow" } ] }Policy that uses fuzzy match
In this example, the source project name can be log-project-dev-a, log-project-dev-b, or log-project-dev-c, and the source Logstore name can be app_a_log, app_b_log, or app_c_log. Replace the project and Logstore names based on your business requirements.
{ "Version": "1", "Statement": [ { "Action": [ "log:ListShards", "log:GetCursorOrData", "log:GetConsumerGroupCheckPoint", "log:UpdateConsumerGroup", "log:ConsumerGroupHeartBeat", "log:ConsumerGroupUpdateCheckPoint", "log:ListConsumerGroup", "log:CreateConsumerGroup" ], "Resource": [ "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log", "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log/*" ], "Effect": "Allow" } ] }For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.
Policy name
Enter a name for the policy. In this example, enter log-etl-source-reader-1-policy.
Attach the policy to the RAM role.
In the left-side navigation pane, choose .
On the Roles page, find the RAM role and click Add Permissions in the Actions column.
Click Custom Policy in the Select Policy section. Then, select the policy that you created in Step 2 and click OK. In this example, the log-etl-source-reader-1-policy policy is selected.
Confirm the authorization result. Then, click Complete.
Obtain the Alibaba Cloud Resource Name (ARN) of the RAM role.
In the Basic Information section of the RAM role, view the ARN of the RAM role. Example: acs:ram::13234:role/logsource.
Grant the RAM role the permissions to write data to a destination Logstore within the same Alibaba Cloud account
If the source and destination Logstores belong to the same Alibaba Cloud account, you can use the Alibaba Cloud account to grant write permissions to the RAM role. Then, the RAM role can write transformed data to the destination Logstore. When you create a data transformation job, you can use the RAM role. For more information, see Create a data transformation job.
Log on to the RAM console by using your Alibaba Cloud account.
Create a policy.
In the left-side navigation pane, choose .
On the Policies page, click Create Policy.
On the Create Policy page, configure the following information and click OK.
Configuration item
Description
Configuration mode
Click JSON.
Policy document
Replace the content in the editor with the following script.
Policy that uses exact match
In this example, the destination project name is log-project-prod, and the destination Logstore name is access_log_output. Replace the project and Logstore names based on your business requirements.
{ "Version": "1", "Statement": [ { "Action": [ "log:Post*", "log:BatchPost*" ], "Resource": "acs:log:*:*:project/log-project-prod/logstore/access_log_output", "Effect": "Allow" } ] }Policy that uses fuzzy match
In this example, the destination project name can be log-project-dev-a, log-project-dev-b, or log-project-dev-c, and the destination Logstore name can be app_a_log_output, app_b_log_output, or app_c_log_output. Replace the project and Logstore names based on your business requirements.
{ "Version": "1", "Statement": [ { "Action": [ "log:Post*", "log:BatchPost*" ], "Resource": "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log_output", "Effect": "Allow" } ] }For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.
Policy name
Enter a name for the policy. In this example, enter log-etl-target-writer-1-policy.
Attach the policy to the RAM role.
In the left-side navigation pane, choose .
On the Roles page, find the RAM role and click Add Permissions in the Actions column.
Click Custom Policy in the Select Policy section. Then, select the policy that you created in Step 2 and click OK. In this example, the log-etl-target-writer-1-policy policy is selected.
Confirm the authorization result. Then, click Complete.
Obtain the ARN of the RAM role.
In the Basic Information section of the RAM role, view the ARN of the RAM role. Example: acs:ram::13234:role/logtarget.
Grant the RAM role the permissions to write data to a destination Logstore across Alibaba Cloud accounts
If the source and destination Logstores belong to different Alibaba Cloud accounts, perform the following steps to grant permissions to the RAM role. For example, a data transformation job is created to read data from a source Logstore that belongs to Alibaba Cloud Account A and write transformed data to a destination Logstore that belongs to Alibaba Cloud Account B.
Before you perform the following steps, make sure that you granted the RAM role the permissions to write data to a destination Logstore within Alibaba Cloud Account B. For more information, see Grant the RAM role the permissions to write data to a destination Logstore within the same Alibaba Cloud account.
Log on to the RAM console by using Alibaba Cloud Account B.
In the left-side navigation pane, choose .
On the Roles page, find the RAM role that you want to manage.
On the Trust Policy Management tab, click Edit Trust Policy.
Modify the trust policy.
Add ID of Alibaba Cloud Account A to which the source Logstore belongs to the Service element. Replace ID of Alibaba Cloud Account A to which the source Logstore belongs with the ID of your Alibaba Cloud account. You can view the ID of your Alibaba Cloud account in the Account Management console. The following policy allows Alibaba Cloud Account A to manage the cloud resources of Alibaba Cloud Account B by using a temporary token.
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "ID of Alibaba Cloud Account A to which the source Logstore belongs@log.aliyuncs.com" ] } } ], "Version": "1" }Obtain the ARN of the RAM role.
In the Basic Information section of the RAM role, view the ARN of the RAM role. Example: acs:ram::13234:role/logtarget.
What to do next
You can enter the ARN of the RAM role in a data transformation job. For more information, see Create a data transformation job.
In Section 1, enter the ARN of the RAM role that has permissions to read data from a source Logstore. For more information, see Grant the RAM role the permissions to read data from a source Logstore.
In Section 2, enter the ARN of the RAM role that has permissions to write data to a destination Logstore. For more information, see Grant the RAM role the permissions to write data to a destination Logstore within the same Alibaba Cloud account or Grant the RAM role the permissions to write data to a destination Logstore across Alibaba Cloud accounts.
