Prerequisites
A Resource Access Management (RAM) role is created. For more information, see Create a RAM role for a trusted Alibaba Cloud service.
Important
When you create a RAM role, you must set the Select Trusted Entity parameter to Alibaba Cloud Service and the Select Trusted Service parameter to Log Service.
Check the trust policy of the RAM role. Make sure that the Service element contains at least "log.aliyuncs.com".
{
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": [
"log.aliyuncs.com"
]
}
}
],
"Version": "1"
}
Grant the RAM role the read-only permissions on a source logstore
After you use an Alibaba Cloud account to grant the RAM role the read-only permissions on a source logstore, the RAM role can read data from the source logstore. When you create a data transformation job, you can use the RAM role. For more information, see Create a data transformation job.
Log on to the RAM console by using your Alibaba Cloud account or a RAM user who has administrative rights.
Create a custom policy. In this example, the log-etl-source-reader-policy policy is created. The policy grants the read-only permissions on a source logstore. For more information, see Create a custom policy on the JSON tab.
On the JSON tab of the Create Policy page, you can use the policy document that uses exact match or fuzzy match for authorization to replace the existing script in the code editor.
Exact match for authorization
Fuzzy match for authorization
In this example, the source project name is log-project-prod, and the source logstore name is access_log. Replace the project and logstore names based on your business requirements.
{
"Version": "1",
"Statement": [
{
"Action": [
"log:ListShards",
"log:GetCursorOrData",
"log:GetConsumerGroupCheckPoint",
"log:UpdateConsumerGroup",
"log:ConsumerGroupHeartBeat",
"log:ConsumerGroupUpdateCheckPoint",
"log:ListConsumerGroup",
"log:CreateConsumerGroup"
],
"Resource": [
"acs:log:*:*:project/log-project-prod/logstore/access_log",
"acs:log:*:*:project/log-project-prod/logstore/access_log/*"
],
"Effect": "Allow"
}
]
}
In this example, the source project names are log-project-dev-a, log-project-dev-b, and log-project-dev-c, and the source logstore names are app_a_log, app_b_log, and app_c_log. Replace the project and logstore names based on your business requirements.
{
"Version": "1",
"Statement": [
{
"Action": [
"log:ListShards",
"log:GetCursorOrData",
"log:GetConsumerGroupCheckPoint",
"log:UpdateConsumerGroup",
"log:ConsumerGroupHeartBeat",
"log:ConsumerGroupUpdateCheckPoint",
"log:ListConsumerGroup",
"log:CreateConsumerGroup"
],
"Resource": [
"acs:log:*:*:project/log-project-dev-*/logstore/app_*_log",
"acs:log:*:*:project/log-project-dev-*/logstore/app_*_log/*"
],
"Effect": "Allow"
}
]
}
Attach the created custom policy to the RAM role. For more information, see Grant permissions to a RAM role.
Grant the RAM role the write permissions on a destination logstore within the same Alibaba Cloud account
If the source and destination logstores belong to the same Alibaba Cloud account, you can use the Alibaba Cloud account to grant the write permissions to the RAM role. Then, the RAM role can write transformed data to the destination logstore. When you create a data transformation job, you can use the RAM role. For more information, see Create a data transformation job.
Create a custom policy. In this example, the log-etl-target-writer-policy policy is created. The policy grants the permissions to write transformation results to a destination logstore. For more information, see Create a custom policy on the JSON tab.
On the JSON tab of the Create Policy page, you can use the policy document that uses exact match or fuzzy match for authorization to replace the existing script in the code editor.
Exact match for authorization
Fuzzy match for authorization
In this example, the destination project name is log-project-prod, and the destination logstore name is access_log_output. Replace the project and logstore names based on your business requirements.
{
"Version": "1",
"Statement": [
{
"Action": [
"log:Post*",
"log:BatchPost*"
],
"Resource": "acs:log:*:*:project/log-project-prod/logstore/access_log_output",
"Effect": "Allow"
}
]
}
In this example, the destination project names are log-project-dev-a, log-project-dev-b, and log-project-dev-c, and the destination logstore names are app_a_log_output, app_b_log_output, and app_c_log_output. Replace the project and logstore names based on your business requirements.
{
"Version": "1",
"Statement": [
{
"Action": [
"log:Post*",
"log:BatchPost*"
],
"Resource": "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log_output",
"Effect": "Allow"
}
]
}
Attach the created custom policy to the RAM role. For more information, see Grant permissions to a RAM role.
Grant the RAM role the write permissions on a destination logstore across Alibaba Cloud accounts
If the source and destination logstores belong to different Alibaba Cloud accounts, perform the following steps to grant permissions to the RAM role. For example, a data transformation job is created to read data from a source logstore that belongs to Alibaba Cloud Account A and write transformed data to a destination logstore that belongs to Alibaba Cloud Account B.
Log on to the RAM console by using your Alibaba Cloud account or a RAM user who has administrative rights.
In the left-side navigation pane, choose .
On the Roles page, find the RAM role that you want to manage and click the role name.
On the Trust Policy tab, click Edit Trust Policy.
Add ID of Alibaba Cloud Account A to which the source logstore belongs to the Service element. Replace ID of Alibaba Cloud Account A to which the source logstore belongs with the ID of your Alibaba Cloud account. You can view the ID of your Alibaba Cloud account in the Account Management console. The following policy allows Alibaba Cloud Account A to manage the cloud resources of Alibaba Cloud Account B by using a temporary token.
{
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": [
"ID of Alibaba Cloud Account A to which the source logstore belongs@log.aliyuncs.com"
]
}
}
],
"Version": "1"
}
Obtain the Alibaba Cloud Resource Name (ARN) of the RAM role. For more information, see View the information about a RAM role.
What to do next
You can specify the Alibaba Cloud Resource Name (ARN) of the RAM role for a data transformation job. For more information, see Create a data transformation job.
Elastic Compute Service (ECS)
Container Compute Service (ACS)
