Access Simple Log Service SDK on an ECS instance without the need for an AccessKey pair - Simple Log Service

To access resources of Alibaba Cloud services other than Simple Log Service, Elastic Compute Service (ECS) instances or applications that are deployed on the ECS instances must have the required access credentials. Alibaba Cloud services use the access credentials to authenticate the identities and permissions of the ECS instances or the applications. You can attach an instance Resource Access Management (RAM) role to an ECS instance. The ECS instance and the applications that are deployed on the ECS instance can automatically obtain and refresh temporary access credentials from within the instance. This way, the ECS instance and the applications can access the resources of Alibaba Cloud services other than Simple Log Service without the need to expose AccessKey pairs. This reduces the risk of AccessKey pair leaks and allows RAM role-based, fine-grained management of access permissions on resources to prevent excessive permissions from being granted. This topic describes how to access Simple Log Service SDK on an ECS instance without the need for an AccessKey pair.

Note

An instance RAM role is a type of RAM role whose trusted entity is an Alibaba Cloud service. This type of RAM role is used to grant access across Alibaba Cloud services and can be assumed by Alibaba Cloud services. For more information about RAM roles, see What is a RAM role?
An Alibaba Cloud AccessKey pair is a secure identity credential that you can use to access Alibaba Cloud resources by calling API operations. You can use an AccessKey pair to sign API requests to pass the security authentication. For more information about AccessKey pairs, see AccessKey pair.
For more information about custom policies for Simple Log Service, see Examples of using custom policies to grant permissions to a RAM user.

Limits

You can only access Simple Log Service SDK for Java or Simple Log Service SDK for Go on an ECS instance without the need for an AccessKey pair.
The ECS instances to which you want to attach instance RAM roles must reside in virtual private clouds (VPCs).
You can attach only one instance RAM role to an ECS instance.

Step 1: Create an instance RAM role and attach the role to an ECS instance

Use the console

Log on to the RAM console to create an instance RAM role.
Create a RAM role whose trusted entity is an Alibaba Cloud service.
In the left-side navigation pane, choose Identities > Roles. On the Roles page, click Create Role. On the Create Role page, set the following parameters to specific values and configure other parameters based on your business requirements. For more information about the parameter settings, see the Create a regular service role.
- Select Trusted Entity: Select Alibaba Cloud Service.
- Role Type: Select Normal Service Role.
- Select Trusted Service: Select Elastic Compute Service.
Grant permissions to the instance RAM role.
Attach system policies or custom policies that you create to the instance RAM role to grant the instance RAM role the permissions to access or manage specific resources. For example, you can use the AliyunLogReadOnlyAccess policy to grant the permissions to read data from Simple Log Service.
Note
You can attach system policies or custom policies to the instance RAM role. If system policies do not meet your business requirements, you can create custom policies. For more information, see Create custom policies.
Attach the instance RAM role to an ECS instance.
1. Log on to the ECS console.
2. In the left-side navigation pane, choose Instances & Images > Instances.
3. In the top navigation bar, select the region and resource group to which the resource belongs.
4. Find the ECS instance to which you want to attach the instance RAM role and choose > Instance Settings > Attach/Detach RAM Role in the Actions column.
5. In the Attach/Detach RAM Role dialog box, select the instance RAM role that you create from the RAM Role drop-down list and click Confirm.

Use API operations

Create and configure an instance RAM role.

Call the CreateRole operation to create an instance RAM role.

Set the AssumeRolePolicyDocument parameter to the following policy:

{
     "Statement": [
     {
         "Action": "sts:AssumeRole",
         "Effect": "Allow",
         "Principal": {
         "Service": [
         "ecs.aliyuncs.com"
         ]
         }
     }
     ],
     "Version": "1"
 }

Optional. Call the CreatePolicy operation to create a policy.

If you already have a policy that can be attached to the instance RAM role, skip this step.

Set the PolicyDocument parameter to the following policy:

{
     "Statement": [
         {
         "Action": [
             "oss:Get*",
             "oss:List*"
         ],
         "Effect": "Allow",
         "Resource": "*"
         }
     ],
     "Version": "1"
 }

Call the AttachPolicyToRole operation to attach the policy to the instance RAM role.
Call the AttachInstanceRamRole operation to attach the instance RAM role to an ECS instance.

Step 2: Access Simple Log Service SDK

Java

Simple Log Service SDK for Java

To use Simple Log Service SDK for Java in a Maven project, you need to only add the related dependencies to the pom.xml file. For more information, see Overview of Simple Log Service SDK for Java.
Important
Make sure that the SDK version is 0.6.110 or later.
```
<dependency>
    <groupId>com.aliyun.openservices</groupId>
    <artifactId>aliyun-log</artifactId>
    <version>0.6.110</version>
</dependency>
```

Specify the created instance RAM role for the Client class. For more information, see Step 1: Create an instance RAM role and attach the role to an ECS instance.

import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.response.GetProjectResponse;

public class Main {
  public static void main(String[] args) throws LogException {
    // Set the RoleName parameter to the created instance RAM role.
    Client client = new Client("cn-hangzhou.log.aliyuncs.com", "RoleName");
    GetProjectResponse resp = client.GetProject("your-project");
    System.out.println(resp.GetProjectDescription());
  }
}

Aliyun Log Java Producer

To use Aliyun Log Java Producer in a Maven project, you need to only add the related dependencies to the pom.xml file. For more information, see Aliyun Log Java Producer.
Important
Make sure that the Producer version is 0.3.22 or later.
```
<dependency>
    <groupId>com.aliyun.openservices</groupId>
    <artifactId>aliyun-log-producer</artifactId>
    <version>0.3.22</version>
</dependency>
```

Specify the created instance RAM role for the ProjectConfig parameter in the project settings. For more information, see Step 1: Create an instance RAM role and attach the role to an ECS instance.

import com.aliyun.openservices.aliyun.log.producer.*;
import com.aliyun.openservices.aliyun.log.producer.errors.ProducerException;
import com.aliyun.openservices.log.common.LogItem;
import com.aliyun.openservices.log.common.auth.CredentialsProvider;
import com.aliyun.openservices.log.common.auth.ECSRoleCredentialsProvider;

public class ExampleUsage {

    public static void main(String[] args) throws ProducerException, InterruptedException {
        String endpoint = "cn-hangzhou.log.aliyuncs.com";
        String project = "your-project";
        String logStore = "your-logstore";
        // Configure the ProducerConfig parameter.
        Config producerConfig = new ProducerConfig();
        // Create a producer.
        Producer producer = new LogProducer(new ProducerConfig());
        // Replace your-ecs-ram-role-name with the created instance RAM role.
        CredentialsProvider provider = new ECSRoleCredentialsProvider("your-ecs-ram-role-name");
        // Configure a project.
        ProjectConfig projectConfig = new ProjectConfig(project, endpoint, provider, null);
        producer.putProjectConfig(projectConfig);

        // send logs
        producer.send(project, logStore, buildLogItem());
        producer.send(project, logStore, null, null, buildLogItem());
        producer.send(project, logStore, "", "", buildLogItem());
        producer.close();
    }

    public static LogItem buildLogItem() {
        LogItem logItem = new LogItem();
        logItem.PushBack("k1", "v1");
        logItem.PushBack("k2", "v2");
        return logItem;
    }
}

Aliyun Log Java Consumer

To use Aliyun Log Java Consumer in a Maven project, you need to only add the related dependencies to the pom.xml file. For more information, see Use consumer groups to consume logs.
Important
Make sure that the Consumer version is 0.6.47 or later.
```
<dependency>
  <groupId>com.aliyun.openservices</groupId>
  <artifactId>loghub-client-lib</artifactId>
  <version>0.6.47</version>
</dependency>
```

Specify the created instance RAM role for the LogHubConfig parameter to create a consumer. For more information, see Step 1: Create an instance RAM role and attach the role to an ECS instance.

import com.aliyun.openservices.log.common.auth.ECSRoleCredentialsProvider;
import com.aliyun.openservices.log.common.auth.CredentialsProvider;
import com.aliyun.openservices.loghub.client.ClientWorker;
import com.aliyun.openservices.loghub.client.config.LogHubConfig;
import com.aliyun.openservices.loghub.client.exceptions.LogHubClientWorkerException;

public class Main {
    private static String Endpoint = "cn-hangzhou.log.aliyuncs.com";
    private static String Project = "your-project";
    private static String Logstore = "your-logstore";
    private static String ConsumerGroup = "consumerGroupX";

    public static void main(String[] args) throws LogHubClientWorkerException, InterruptedException {
        // Replace your-ecs-ram-role-name with the created instance RAM role.
        CredentialsProvider provider = new ECSRoleCredentialsProvider("your-ecs-ram-role-name");

        LogHubConfig config = new LogHubConfig(ConsumerGroup, "consumer_1", Endpoint, Project, Logstore, provider,
                LogHubConfig.ConsumePosition.BEGIN_CURSOR);
        config.setMaxFetchLogGroupSize(1000);

        ClientWorker worker = new ClientWorker(new SampleLogHubProcessorFactory(), config);
        Thread thread = new Thread(worker);
        thread.start();
        Thread.sleep(60 * 60 * 1000);
        worker.shutdown();
        Thread.sleep(30 * 1000);
    }
}

Go

Important

Make sure that the version of Simple Log Service SDK for Go is 0.1.83 or later. For more information about how to upgrade Simple Log Service SDK for Go, see Install or upgrade Simple Log Service SDK for Go.

Simple Log Service for Go

Specify the created instance RAM role for the provider parameter. For more information, see Step 1: Create an instance RAM role and attach the role to an ECS instance.

package main

import (
	"fmt"

	sls "github.com/aliyun/aliyun-log-go-sdk"
)

func main() {
        // The Simple Log Service endpoint. In this example, the Simple Log Service endpoint for the China (Hangzhou) region is used. Replace the parameter value with the actual endpoint. 
	Endpoint := "cn-chengdu.log.aliyuncs.com"
	// Replace your-ecs-ram-role-name with the created instance RAM role.
	provider := sls.NewEcsRamRoleCredentialsProvider("your-ecs-ram-role-name")
	// Create a Simple Log Service client. 
	client := sls.CreateNormalInterfaceV2(Endpoint, provider)
	resp, err := client.GetProject("your-project")
	if err != nil {
		panic(err)
	}
	fmt.Println(resp.Description)
}

Aliyun Log Go Producer

Specify the created instance RAM role for the provider parameter in the ProducerConfig settings. For more information, see Step 1: Create an instance RAM role and attach the role to an ECS instance.

package main

import (
	"fmt"
	"os"
	"os/signal"
	"sync"
	"time"

	sls "github.com/aliyun/aliyun-log-go-sdk"
	"github.com/aliyun/aliyun-log-go-sdk/producer"
	"github.com/gogo/protobuf/proto"
)

func main() {
	producerConfig := producer.GetDefaultProducerConfig()
	producerConfig.Endpoint = "cn-hangzhou.log-aliyuncs.com"
    // Replace your-ecs-ram-role-name with the created instance RAM role.
	provider := sls.NewEcsRamRoleCredentialsProvider("your-ecs-ram-role-name")
	producerConfig.CredentialsProvider = provider
	producerInstance := producer.InitProducer(producerConfig)
	ch := make(chan os.Signal)
	signal.Notify(ch, os.Kill, os.Interrupt)
	producerInstance.Start()
    
	var m sync.WaitGroup
	for i := 0; i < 10; i++ {
		m.Add(1)
		go func() {
			defer m.Done()
			for i := 0; i < 1000; i++ {
				log := producer.GenerateLog(uint32(time.Now().Unix()), map[string]string{"content": "test", "content2": fmt.Sprintf("%v", i)})
				err := producerInstance.SendLog("log-project", "log-store", "topic", "127.0.0.1", log)
				if err != nil {
					fmt.Println(err)
				}
			}
		}()
	}
	m.Wait()
	fmt.Println("Send completion")
	if _, ok := <-ch; ok {
		fmt.Println("Get the shutdown signal and start to shut down")
		producerInstance.Close(60000)
	}
}

Aliyun Log Go Consumer

Specify the created instance RAM role for the CredentialsProvider parameter. For more information, see Step 1: Create an instance RAM role and attach the role to an ECS instance.

package main

import (
	"fmt"
	"os"
	"os/signal"
	"syscall"

	sls "github.com/aliyun/aliyun-log-go-sdk"
	consumerLibrary "github.com/aliyun/aliyun-log-go-sdk/consumer"
	"github.com/go-kit/kit/log/level"
)

func main() {
	option := consumerLibrary.LogHubConfig{
		Endpoint: "cn-hangzhou.log-aliyuncs.com",
		// Replace your-ecs-ram-role-name with the created instance RAM role.
		CredentialsProvider: sls.NewEcsRamRoleCredentialsProvider("your-ecs-ram-role-name"),
		Project:             "your-project",
		Logstore:            "your-logstore",
		ConsumerGroupName:   "your-consumer-group",
		ConsumerName:        "your-consumer-group-consumer-1",
		CursorPosition:      consumerLibrary.END_CURSOR,
	}

	consumerWorker := consumerLibrary.InitConsumerWorkerWithCheckpointTracker(option, process)
	ch := make(chan os.Signal, 1)
	signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM)
	consumerWorker.Start()
	if _, ok := <-ch; ok {
		level.Info(consumerWorker.Logger).Log("msg", "get stop signal, start to stop consumer worker", "consumer worker name", option.ConsumerName)
		consumerWorker.StopAndWait()
	}
}

// The data consumption logic.
func process(shardId int, logGroupList *sls.LogGroupList, checkpointTracker consumerLibrary.CheckPointTracker) (string, error) {
	fmt.Println(shardId, logGroupList)
	checkpointTracker.SaveCheckPoint(false)
	return "", nil
}

References

If your self-managed application is deployed on an ECS instance, you can attach an instance RAM role to the ECS instance and use the instance RAM role to access Key Management Service (KMS) from the instance. For more information, see Use the instance RAM role attached to an ECS instance to securely access KMS.
If an ECS instance no longer requires specific permissions, you can revoke the permissions from the instance RAM role that is attached to the instance. For more information, see Revoke permissions from a RAM role.
If you hard code a plaintext AccessKey pair in the code that you use to call API operations of Alibaba Cloud, the AccessKey pair may be leaked due to improper permission management of the code repository. To call the API operations of Alibaba Cloud, we recommend that you use access credentials instead of a hard-coded AccessKey pair. For more information, see Best practices for using an access credential to call API operations and Credential security solutions.