How to access data by using AccessKey pairs - Simple Log Service

Simple Log Service allows you to use a default role, a custom role, or an AccessKey pair to create a data transformation job. We recommend that you do not use the AccessKey pair of an Alibaba Cloud account to create a data transformation job because the Alibaba Cloud account has the highest permissions. You can use the AccessKey pair of a Resource Access Management (RAM) user to create a data transformation job. This topic describes how to grant a RAM user the permissions to access data in a logstore.

Prerequisites

A RAM user is created. For more information, see Create a RAM user.

Important

When you create a RAM user, set the Access Mode parameter to Using permanent AccessKey to access and record the AccessKey pair of the RAM user.
The AccessKey secret of a RAM user is displayed only when you create the AccessKey pair for the RAM user. You cannot query the AccessKey secret after you create the AccessKey pair. Store your AccessKey secret in a secure location.

Grant a RAM user the read-only permissions on a source logstore

After you grant a RAM user the read-only permissions on a source logstore, the RAM user can read data from the source logstore. When you create a data transformation job, you can enter the AccessKey pair of the RAM user. For more information, see Create a data transformation job.

Log on to the RAM console by using your Alibaba Cloud account or a RAM user who has administrative rights.

Create a custom policy. In this example, the log-etl-source-reader-policy policy is created. The policy grants the read-only permissions on a source logstore. For more information, see Create a custom policy on the JSON tab.

On the JSON tab of the Create Policy page, you can use the policy document that uses exact match or fuzzy match for authorization to replace the existing script in the code editor.

Exact match for authorization

In this example, the source project name is log-project-prod, and the source logstore name is access_log. Replace the project and logstore names based on your business requirements.

{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:ListShards",
        "log:GetCursorOrData",
        "log:GetConsumerGroupCheckPoint",
        "log:UpdateConsumerGroup",
        "log:ConsumerGroupHeartBeat",
        "log:ConsumerGroupUpdateCheckPoint",
        "log:ListConsumerGroup",
        "log:CreateConsumerGroup"
      ],
      "Resource": [
        "acs:log:*:*:project/log-project-prod/logstore/access_log",
        "acs:log:*:*:project/log-project-prod/logstore/access_log/*"
      ],
      "Effect": "Allow"
    }
  ]
}

Fuzzy match for authorization

In this example, the source project names are log-project-dev-a, log-project-dev-b, and log-project-dev-c, and the source logstore names are app_a_log, app_b_log, and app_c_log. Replace the project and logstore names based on your business requirements.

{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:ListShards",
        "log:GetCursorOrData",
        "log:GetConsumerGroupCheckPoint",
        "log:UpdateConsumerGroup",
        "log:ConsumerGroupHeartBeat",
        "log:ConsumerGroupUpdateCheckPoint",
        "log:ListConsumerGroup",
        "log:CreateConsumerGroup"
      ],
      "Resource": [
        "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log",
    "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log/*"
      ],
      "Effect": "Allow"
    }
  ]
}

Attach the created custom policy to the RAM user. For more information, see Grant permissions to a RAM user.

Grant a RAM user the write permissions on a destination logstore

After you grant a RAM user the write permissions on a destination logstore, the RAM user can write transformation results to the destination logstore. When you create a data transformation job, you can enter the AccessKey pair of the RAM user. For more information, see Create a data transformation job.

Create a custom policy. In this example, the log-etl-target-writer-policy policy is created. The policy grants the permissions to write transformation results to a destination logstore. For more information, see Create a custom policy on the JSON tab.
On the JSON tab of the Create Policy page, you can use the policy document that uses exact match or fuzzy match for authorization to replace the existing script in the code editor.
Exact match for authorization
In this example, the destination project name is log-project-prod, and the destination logstore name is access_log_output. Replace the project and logstore names based on your business requirements.
```
{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:Post*",
        "log:BatchPost*"
      ],
       "Resource": "acs:log:*:*:project/log-project-prod/logstore/access_log_output",
      "Effect": "Allow"
    }
  ]
}
```
Fuzzy match for authorization
In this example, the destination project names are log-project-dev-a, log-project-dev-b, and log-project-dev-c, and the destination logstore names are app_a_log_output, app_b_log_output, and app_c_log_output. Replace the project and logstore names based on your business requirements.
```
{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:Post*",
        "log:BatchPost*"
      ],
       "Resource": "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log_output",
      "Effect": "Allow"
    }
  ]
}
```
Attach the created custom policy to the RAM user. For more information, see Grant permissions to a RAM user.

What to do next

You can specify the AccessKey pair of a RAM user for a data transformation job. For more information, see Create a data transformation job.