Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Using Project Policies to Isolate iLogtail Write Permissions in Different Networks

Using Project Policies to Isolate iLogtail Write Permissions in Different Networks

This article describes three methods for using project policies to control the data source reported by iLogtail.

By Xunfei, from Alibaba Cloud Storage Team

Target Readers

DevOps engineers for digital systems, Site Reliability Engineers (SRE), and operational staff for observability platforms.

Background

By default, the Simple Log Service (SLS) project allows data to be written from any IP address. Although Resource Access Management (RAM) users can regulate permissions, this control is not possible when writing data to SLS with iLogtail, due to the absence of a corresponding RAM user entity. In such scenarios, you must employ a project policy to govern the write permissions of iLogtail. For instance, if you have a production cluster A that is running steadily online, you can collect logs using iLogtail and write them to Project A, setting up automated operation and maintenance (O&M) policies like alerts. To avoid accidental writing of logs from test clusters or new clusters into Project A, which could disrupt daily O&M activities, a project policy can be utilized to manage the write permissions for iLogtail.

Terms

iLogtail

iLogtail is a log collection agent provided by SLS. It is used to gather logs from various servers, including Alibaba Cloud ECS instances, self-built IDCs, and other cloud vendors.

Project Policy

Project Policy is an authorization policy introduced by SLS specifically for projects. It allows you to define access control policies for users to manage specific SLS resources. For more information, see https://www.alibabacloud.com/help/en/sls/use-project-policies-to-manage-access-permissions-on-log-service-resources

Note:

• If you set the Principal element to an asterisk (*) and do not configure the Condition element when you configure a project policy, the policy applies to all users except the project owner; If you set the Principal element to an asterisk (*) and configure the Condition element when you configure a project policy, the policy applies to all users, including the project owner.

Based on the above description, if a condition is included, the policy will apply to all users, including the project owner, and will therefore be effective for iLogtail writing. Currently, project policies can only be configured using SDKs, as there is no web console available at the moment.

Scenarios and Solutions

Case 1: Allow Writing from Specific VPCs

Step 1: Verify that two different VPC networks can write.

The IP addresses and VPCs of the two test machines are as follows:

Cluster IP VPC
A 172.17.243.174 vpc-0jl*jg5
B 10.0.0.243 vpc-0jl*yy2

Configure iLogtail to collect nginx.log and write mock data to nginx.log. The following figure shows the query result in the SLS console.

1

It proves that both VPCs can write.

Step 2: Configure the project policy.

Use the Java SDK to configure a project policy. It takes about 1 minute for the policy to take effect.

1.  Clone the code library of the Java SDK.

git@gitlab.alibaba-inc.com:sls/aliyun-log-java-sdk.git

2.  Create src/main/java/com/aliyun/openservices/log/sample/ProjectPolicyDemo.java

3.  Open IDE, delete the scope of junit in pom.xml, and paste the following code:

2

package com.aliyun.openservices.log.sample; import com.aliyun.openservices.log.Client; import com.aliyun.openservices.log.exception.LogException; import org.junit.Assert; public class ProjectPolicyDemo {// In this example, the AccessKey ID and AccessKey secret are obtained from environment variables. static String accessKeyId=System.getenv ("ALIBABA_CLOUD_ACCESS_KEY_ID"); static String accessKey=System.getenv ("ALIBABA_CLOUD_ACCESS_KEY_SECRET"); static String endPoint = "cn-wulanchabu.log.aliyuncs.com"; static String projectName = "test-project"; static Client client=new Client(endPoint, accessKeyId, accessKey); public static void main(String[] args) throws LogException { try { client.GetProject(projectName); } catch (LogException e) { Assert.fail("should not fail : " + e.GetErrorCode()); } String policyText = "{ \"Version\": \"1\",\n" + " \"Statement \: [{" + " \"Action\": [\"log:PostLogstoreLogs\", \"log:BatchPostLogStoreLogs\"]," + " \"Resource\": \"*\",\n + " \"Condition\": { \"StringNotEquals\": { \"acs:SourceVpc\": [" vpc-0jl***************jg5 "]}}," + "" Effect ": " Deny "}] }"; client.setProjectPolicy(projectName, policyText); client.getProjectPolicy(projectName); Assert.assertEquals(policyText, client.getProjectPolicy(projectName).getPolicyText()) ;}}

The preceding policy indicates that for data writing action of log:PostLogstoreLogs and log:BatchPostLogStoreLogs, if the source does not match the specified VPC vpc-0jl*jg5, the action is denied.

4.  Execute the code. Edit Run Configuration, configure the environment variables of AK and SK, and save and execute the code.

3

The execution is successful if the following message appears in the console: Process finished with exit code 0.

Step 3: Verify that the specified VPC network can write and other networks cannot

Send test data again through the two test machines. The following table describes the query results in the SLS console.

4

The following figure shows that machines in cluster A vpc-0jl*jg5 can write, while the machines in other networks cannot.

The Unauthorized error message is displayed in Cloud Lens For SLS.

5
6

On the host, the following error message is reported in ilogtail.LOG, which proves that the project policy takes effect: StatusCode:401 ErrorCode:Unauthorized ErrorMessage:Access denied by project policy.

[2023-11-06 17:59:39.075256]    [warning]       [001528]        /build/logtail/sender/Sender.cpp:372            SendFail:write unauthorized     Operation:discard data  Suggestion:check https connection to endpoint or access keys provided   RequestId:6548B90B1D6AD57F74241836      StatusCode:401  ErrorCode:Unauthorized  ErrorMessage:Access denied by project policy.   ResponseTime:0  Region:cn-wulanchabu    Project:xunfei-test-wlcb        Logstore:ecs_logtail_log        Config:##1.0##xunfei-test-wlcb$verylonglog      RetryTimes:6    TotalSendCost:0 LogLines:1      Bytes:209       Endpoint:http://cn-wulanchabu-intranet.log.aliyuncs.com IsProfileData:false

Additional Steps: Delete the Project Policy.

If you need to restore the write capability from public IP addresses, you can delete the project policy.

package com.aliyun.openservices.log.sample;

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.exception.LogException;
import org.junit.Assert;

public class ProjectPolicyDemo {
  // In this example, the AccessKey ID and AccessKey secret are obtained from environment variables.
  static String accessKeyId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
  static String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
  static String endPoint = "cn-wulanchabu.log.aliyuncs.com";
  static String projectName = "test-project";
  static Client client = new Client(endPoint, accessKeyId, accessKey);

  public static void main(String[] args) throws LogException {
    client.deleteProjectPolicy(projectName);
    Assert.assertEquals("", client.getProjectPolicy(projectName).getPolicyText());
  }
}

Case 2: Prohibit Writing from Public IP Addresses

Step 1: Verify that both private and public IP addresses can write data.

Enable the logstore to record public IP addresses to distinguish between private and public write traffic.

7

Use an ECS instance to write data by using a private endpoint, and then switch to a public endpoint to write data. Endpoints can be switched by modifying the ilogtail_config.json. The following figure shows the query result in the SLS console.

8

It proves that both private and public IP addresses can write data.

Step 2: Configure the project policy.

Use the Java SDK to configure a project policy. It takes about 1 minute for the policy to take effect.


package com.aliyun.openservices.log.sample;

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.exception.LogException;
import org.junit.Assert;

public class ProjectPolicyDemo {
  // In this example, the AccessKey ID and AccessKey secret are obtained from environment variables.
  static String accessKeyId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
  static String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
  static String endPoint = "cn-wulanchabu.log.aliyuncs.com";
  static String projectName = "test-project";
  static Client client = new Client(endPoint, accessKeyId, accessKey);

  public static void main(String[] args) throws LogException {
    try {
      client.GetProject(projectName);
    } catch (LogException e) {
      Assert.fail("should not fail : " + e.GetErrorCode());
    }
    String policyText = "{  \"Version\": \"1\",\n" +
        "   \"Statement\": [{" +
        "   \"Action\": [\"log:PostLogstoreLogs\", \"log:BatchPostLogStoreLogs\"]," +
        "   \"Resource\": \"*\",\n" +
        "   \"Condition\": {\"StringNotLike\": {\"acs:SourceVpc\":[\"vpc-*\"]}}," +
        "   \"Effect\": \"Deny\"}] }";
    client.setProjectPolicy(projectName, policyText);
    client.getProjectPolicy(projectName);
    Assert.assertEquals(policyText, client.getProjectPolicy(projectName).getPolicyText());
  }
}

The preceding policy indicates that for data writing action of log:PostLogstoreLogs and log:BatchPostLogStoreLogs, if the source is not the internal network (the source VPC does not match vpc-*), the action is denied.

Step 3: Verify that data from the private IP address can be written but data from the public IP address cannot.

Verify again that data is written by using private and public endpoints.

9

The following figure shows that the private IP address can write, but the public IP address cannot. The Unauthorized error message is displayed in Cloud Lens For SLS.

10
11

On the host, the following error message is reported in ilogtail.LOG, which proves that the project policy takes effect: StatusCode:401 ErrorCode:Unauthorized ErrorMessage:Access denied by project policy.

 [2023-11-06 11:56:29.724337]    [warning]       [3821515]       /build/logtail/sender/Sender.cpp:388            SendFail:write unauthorized     Operation:discard data  Suggestion:check https connection to endpoint or access keys provided   RequestId:6548596170EF2E88A1AE304C      StatusCode:401  ErrorCode:Unauthorized  ErrorMessage:Access denied by project policy.   ResponseTime:0  Region:cn-wulanchabu    Project:xunfei-test-wlcb        Logstore:ecs_logtail_log        Config:##1.0##xunfei-test-wlcb$verylonglog      RetryTimes:6    TotalSendCost:0 LogLines:1      Bytes:250       Endpoint:http://cn-wulanchabu.log.aliyuncs.com  IsProfileData:false

Case 3: Prohibit Writing from Specific IP Addresses

Step 1: Verify that two specific IP addresses can write.

Enable the logstore to record public IP addresses so that the IP addresses can be displayed. For the convenience of our test, we still use the preceding method to obtain two different write IP addresses by switching the private and public endpoints. In this example, the IP address obtained from the private network is 172.17.243.174, and that from the external network is 8.x. x.x. Note that the real IP address of the public network is recorded in __tag__:__client_ip__, while there is no additional record for the private network. The following figure shows the query result in the SLS console.

12

This proves that data from both IP addresses can be written.

Step 2: Configure the Project Policy.

Use the Java SDK to configure a project policy. It takes about 1 minute for the policy to take effect.

package com.aliyun.openservices.log.sample;

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.exception.LogException;
import org.junit.Assert;

public class ProjectPolicyDemo {
  //  In this example, the AccessKey ID and AccessKey secret are obtained from environment variables.
  static String accessKeyId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
  static String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
  static String endPoint = "cn-wulanchabu.log.aliyuncs.com";
  static String projectName = "test-project";
  static Client client = new Client(endPoint, accessKeyId, accessKey);

  public static void main(String[] args) throws LogException {
    try {
      client.GetProject(projectName);
    } catch (LogException e) {
      Assert.fail("should not fail : " + e.GetErrorCode());
    }
    String policyText = "{  \"Version\": \"1\",\n" +
        "   \"Statement\": [{" +
        "   \"Action\": [\"log:PostLogstoreLogs\", \"log:BatchPostLogStoreLogs\"]," +
        "   \"Resource\": \"*\",\n" +
        "   \"Condition\": {\"IpAddress\":{\"acs:SourceIp\":[\"8.0.0.0/8\"]}}," + // 可以写特定IP，这里为了不暴露真实IP写了地址段
        "   \"Effect\": \"Deny\"}] }";
    client.setProjectPolicy(projectName, policyText);
    client.getProjectPolicy(projectName);
    Assert.assertEquals(policyText, client.getProjectPolicy(projectName).getPolicyText());
  }
}

The preceding policy indicates that for data writing action of log:PostLogstoreLogs and log:BatchPostLogStoreLogs, if the source is an address of 8...*, the action is denied.

Step 3: Verify that specific IP addresses cannot write.

Verify again that data is written by using private and public endpoints.

13

The following figure shows that the private IP address can write, but the public IP address 8.x. x.x cannot. The Unauthorized error message is displayed in Cloud Lens For SLS.

14
15

On the host, the following error message is reported in ilogtail.LOG, which proves that the project policy takes effect: StatusCode:401 ErrorCode:Unauthorized ErrorMessage:Access denied by project policy. It also proves that users cannot bypass the policy by configuring working_ip of iLogtail to forge IP addresses.

[2023-11-06 13:56:20.680591]    [warning]       [3825176]       /build/logtail/sender/Sender.cpp:388            SendFail:write unauthorized     Operation:discard data  Suggestion:check https connection to endpoint or access keys provided   RequestId:654880043DDEB62EECB83000      StatusCode:401  ErrorCode:Unauthorized  ErrorMessage:Access denied by project policy.   ResponseTime:0  Region:cn-wulanchabu    Project:xunfei-test-wlcb        Logstore:ecs_logtail_log        Config:##1.0##xunfei-test-wlcb$verylonglog      RetryTimes:6    TotalSendCost:0 LogLines:1      Bytes:250       Endpoint:http://cn-wulanchabu.log.aliyuncs.com  IsProfileData:false

Summary

This article presents three methods for using project policies to control the data source reported by iLogtail. Project policies enable us to manage data collection within a secure range. Additionally, project policies can also control other activities such as Logstore creation and index modification. For more information, see https://www.alibabacloud.com/help/en/sls/use-project-policies-to-manage-access-permissions-on-log-service-resources

For languages and common condition keys of project policies, see https://www.alibabacloud.com/help/en/resource-management/user-guide/languages-of-access-control-policies

About iLogtail

As a data collector provided by Alibaba Cloud's SLS (Simple Log Service), iLogtail is capable of running in various environments, including servers, containers, Kubernetes, and embedded systems. It can collect a wide range of observable data types such as logs, monitoring data, traces, and events. iLogtail has been officially open source. We welcome you to use and contribute to its development.

Disclaimer: The views expressed herein are for reference only and don't necessarily represent the official views of Alibaba Cloud.

Storage Networking Storage Service Best practices iLogtail Project Policy
0 1 0
Share on

Read previous post:

Everything You Need To Know About Databases

Read next post:

OpenKruise × iLogtail: The Best Practice for Managing Sidecar Containers for Observable Data Collection

Alibaba Cloud Community

1,080 posts | 265 followers

You may also like

Comments

Alibaba Cloud Community

1,080 posts | 265 followers

Related Products

More Posts by Alibaba Cloud Community

See All