Collect stdout and stderr from Kubernetes containers in DaemonSet mode - Simple Log Service

This topic describes how to create a Logtail configuration in the Simple Log Service (SLS) console and use the Logtail configuration to collect stdout and stderr from containers in DaemonSet mode (old version).

Prerequisites

The Logtail component is installed. For more information, see Install Logtail components in a Kubernetes cluster.
A Logstore is created in the project that you use to install the Logtail component. For more information, see Create a Logstore.
The container from which you want to collect logs continuously generates logs.
Important
Logtail collects only incremental logs. If a log file on a server is not updated after the applied Logtail configuration is delivered to the server, Logtail does not collect logs from the file. For more information, see Read log files.

Features

Logtail can collect the stdout and stderr generated from a container and upload the collected stdout and stderr together with the metadata of the container to SLS. Logtail provides the following features:

Collects stdout and stderr.
Uses the container label whitelist to specify containers from which logs are collected.
Uses the container label blacklist to specify containers from which logs are not collected.
Uses the environment variable whitelist to specify containers from which logs are collected.
Uses the environment variable blacklist to specify containers from which logs are not collected.
Collects multi-line logs. For example, Logtail can collect Java stack logs.
Automatically associates container metadata that needs to be uploaded together with the collected container logs. The metadata includes container names, image names, pod names, namespaces, and environment variables.
If a container runs in a Kubernetes cluster, Logtail also supports the following features:
- Uses Kubernetes namespaces, pod names, and container names to specify containers from which logs are collected.
- Uses the Kubernetes label whitelist to specify containers from which logs are collected.
- Uses the Kubernetes label blacklist to specify containers from which logs are not collected.
- Automatically associates Kubernetes labels that need to be uploaded together with the collected container logs.

Implementation

Logtail communicates with the domain socket of Docker. Logtail queries all Docker containers and identifies the containers from which stdout and stderr must be collected by using the specified labels and environment variables. Logtail determines the location of the log file in which the collected stdout and stderr are stored based on the container metadata for data collection.

When Logtail collects stdout and stderr from a container, Logtail periodically stores checkpoints to a checkpoint file. If Logtail is stopped and then started, Logtail collects logs from the last stored checkpoint. 实现原理

Limits

You can collect stdout and stderr from Kubernetes containers in DaemonSet mode only if Logtail is of V0.16.0 or later and runs on Linux. For more information about how to view and upgrade the version of Logtail, see Install Logtail on a Linux server.
Logtail can collect data only from the containers that use the Docker engine or containerd engine.
- Docker: Logtail accesses the Docker engine in the /run/docker.sock directory. Make sure that the directory exists and Logtail has the permissions to access the directory.
- containerd: Logtail accesses the containerd engine in the /run/containerd/containerd.sock directory. Make sure that the directory exists and Logtail has the permissions to access the directory.
By default, the last multi-line log that is collected by Logtail is cached for 3 seconds. This prevents the multi-line log from being split into multiple logs due to output latency. You can modify the BeginLineTimeoutMs parameter to change the cache time. We recommend that you do not specify a value that is less than 1000 with millisecond precision. If you specify a value that is less than 1000, an error may occur.
If Logtail detects the die event on a stopped container, Logtail no longer collects stdout or stderr from the container. If collection latency exists, some stdout and stderr that are collected before the container is stopped may be lost.
If a container uses the Docker engine, the logging driver can collect stdout and stderr only in the JSON format.
By default, stdout and stderr that are collected from different containers by using the same Logtail configuration have the same context. In this case, you cannot view the context of stdout and stderr in the SLS console by using the contextual query feature or LiveTail feature. If you want to view the context in the preceding scenario, you must add aggregator configurations. For more information, see Overview of Logtail plug-ins for data processing.
By default, the collected data is stored in the content field. Logtail can process the collected data. For more information, see Overview of Logtail plug-ins for data processing.

Create a Logtail configuration

Form configuration

Log on to the Simple Log Service console. In the Projects section, click the project that you want to manage.
On the Log Storage > Logstores tab, click the logstore that you want to manage.
Click the rightwards arrow to the left of the Logstore name and choose Data Collection > Logtail Configurations. On the Logtail Configuration page, click Add Logtail Configuration.
In the Quick Data Import dialog box, click Kubernetes - Stdout and Stderr - Old Version.
In the Machine Group Configuration step, add the machine group that is automatically created by the Logtail component to the Applied Server Groups section and select it. The name of the machine group is k8s-group-*. Click Next.

In the Logtail Configuration step, configure the parameters described in the following tables.

Global Configurations

Parameter	Description
Configuration Name	The name of the Logtail configuration. The name must be unique in a project. After you create the Logtail configuration, you cannot change the name of the Logtail configuration.
Advanced Parameters	Specifies whether to configure the advanced parameters that are related to global configurations. For more information, see CreateLogtailPipelineConfig.

Input Configurations

Parameter	Description
Standard Output	Specifies whether to collect container stdout. If you turn on Standard Output, Logtail collects container stdout.
Standard Error	Specifies whether to collect container stderr. If you turn on Standard Error, Logtail collects container stderr.
Enable Container Metadata Preview	Specifies whether to enable the preview of container metadata. If you turn on Enable Container Metadata Preview, you can view the container metadata after you create the Logtail configuration, including the matched container information and full container information.
Container Filtering	Logtail version If the version of Logtail is earlier than 1.0.34, you can use only environment variables and container labels to filter containers. If the version of Logtail is 1.0.34 or later, we recommend that you use different levels of Kubernetes information to filter containers. The information includes K8s Pod Name Regular Matching, K8s Namespace Regular Matching, K8s Container Name Regular Matching, and Kubernetes Pod Label Whitelist. Filter conditions Important The conditions used to filter containers. Container labels are retrieved by running the docker inspect command. Container labels are different from Kubernetes labels. For more information, see the "Obtain labels" section of the How do I obtain the labels and environment variables of a Docker container? topic. Environment variables are the same as those configured to start containers. For more information, see the "Obtain environment variables" section of the How do I obtain the labels and environment variables of a Docker container? topic. Kubernetes namespaces and container names can be mapped to container labels. The label for a namespace is `io.kubernetes.pod.namespace`. The label for a container name is `io.kubernetes.container.name`. We recommend that you use the two labels to filter containers. For example, the namespace of a pod is `backend-prod`, and the name of a container in the pod is `worker-server`. If you want to collect the logs of the worker-server container, you can specify `io.kubernetes.pod.namespace : backend-prod` or `io.kubernetes.container.name : worker-server` in the container label whitelist. If the two labels do not meet your business requirements, you can use the environment variable whitelist or the environment variable blacklist to filter containers. K8s Pod Name Regular Matching The pod name that specifies the containers from which stdout and stderr are collected. Regular expression matching is supported. For example, if you specify `^(nginx-log-demo.)$,`, all containers in the pods whose name starts with nginx-log-demo are matched. Note* This parameter is available for Logtail V1.0.34 and later. K8s Namespace Regular Matching The namespace that specifies the containers from which stdout and stderr are collected. Regular expression matching is supported. For example, if you specify `^(default\|nginx)$`, all containers in the nginx and default namespaces are matched. Note This parameter is available for Logtail V1.0.34 and later. K8s Container Name Regular Matching The container name that specifies the containers from which stdout and stderr are collected. Regular expression matching is supported. Kubernetes container names are defined in spec.containers. For example, if you specify `^(container-test)$`, all containers whose name is container-test are matched. Note This parameter is available for Logtail V1.0.34 and later. Container Label Whitelist The container label whitelist that specifies the containers from which stdout and stderr are collected. This parameter is empty by default, which indicates that stdout and stderr are collected from all containers. If you configure a container label whitelist, the Label Name parameter is required and the Label Value parameter is optional. If the LabelValue parameter is empty, containers whose container labels contain the keys specified by LabelKey are matched. If the LabelValue parameter is not empty, containers whose container labels consist of the key-value pairs specified by LabelKey and LabelValue are matched. By default, string matching is performed for the values of the LabelValue parameter. Containers are matched only if the values of the container labels are the same as the values of the LabelValue parameter. If you specify a value that starts with a caret (`^`) and ends with a dollar sign (`$`) for the LabelValue parameter, regular expression matching is performed. For example, if you set the LabelKey parameter to `io.kubernetes.container.name` and set the LabelValue parameter to `^(nginx\|cube)$`, a container named nginx and a container named cube are matched. Key-value pairs are connected by using the OR operator. If a container has a container label that consists of one of the specified key-value pairs, the container is matched. Container Label Blacklist The container label blacklist that specifies the containers from which stdout and stderr are not collected. This parameter is empty by default, which indicates that stdout and stderr are collected from all containers. If you configure a container label blacklist, the Label Name parameter is required and the Label Value parameter is optional. If the LabelValue parameter is empty, containers whose container labels contain the keys specified by LabelKey are filtered out. If the LabelValue parameter is not empty, containers whose container labels consist of the key-value pairs specified by LabelKey and LabelValue are filtered out. By default, string matching is performed for the values of the LabelValue parameter. Containers are matched only if the values of the container labels are the same as the values of the LabelValue parameter. If you specify a value that starts with a caret (`^`) and ends with a dollar sign (`$`) for the LabelValue parameter, regular expression matching is performed. For example, if you set the LabelKey parameter to `io.kubernetes.container.name` and set the LabelValue parameter to `^(nginx\|cube)$`, a container named nginx and a container named cube are matched. Key-value pairs are connected by using the OR operator. If a container has a container label that consists of one of the specified key-value pairs, the container is filtered out. Environment Variable Whitelist The environment variable whitelist that specifies the containers from which stdout and stderr are collected. This parameter is empty by default, which indicates that stdout and stderr are collected from all containers. If you configure an environment variable whitelist, the Environment Variable Name parameter is required and the Environment Variable Value parameter is optional. If the EnvValue parameter is empty, containers whose environment variables contain the keys specified by EnvKey are matched. If the EnvValue parameter is not empty, containers whose environment variables consist of the key-value pairs specified by EnvKey and EnvValue are matched. By default, string matching is performed for the values of the EnvValue parameter. Containers are matched only if the values of the environment variables are the same as the values of the EnvValue parameter. If you specify a value that starts with a caret (`^`) and ends with a dollar sign (`$`) for the EnvValue parameter, regular expression matching is performed. For example, if you set the EnvKey parameter to `NGINX_SERVICE_PORT` and set the EnvValue parameter to `^(80\|6379)$`, containers whose port number is 80 and containers whose port number is 6379 are matched. Key-value pairs are connected by using the OR operator. If a container has an environment variable that consists of one of the specified key-value pairs, the container is matched. Environment Variable Blacklist The environment variable blacklist that specifies the containers from which stdout and stderr are not collected. This parameter is empty by default, which indicates that stdout and stderr are collected from all containers. If you configure an environment variable blacklist, the Environment Variable Name parameter is required and the Environment Variable Value parameter is optional. If the EnvValue parameter is empty, containers whose environment variables contain the keys specified by EnvKey are filtered out. If the EnvValue parameter is not empty, containers whose environment variables consist of the key-value pairs specified by EnvKey and EnvValue are filtered out. By default, string matching is performed for the values of the EnvValue parameter. Containers are matched only if the values of the environment variables are the same as the values of the EnvValue parameter. If you specify a value that starts with a caret (`^`) and ends with a dollar sign (`$`) for the EnvValue parameter, regular expression matching is performed. For example, if you set the EnvKey parameter to `NGINX_SERVICE_PORT` and set the EnvValue parameter to `^(80\|6379)$`, containers whose port number is 80 and containers whose port number is 6379 are matched. Key-value pairs are connected by using the OR operator. If a container has an environment variable that consists of one of the specified key-value pairs, the container is filtered out. Kubernetes Pod Label Whitelist The Kubernetes label whitelist that specifies the containers from which stdout and stderr are collected. If you configure a Kubernetes label whitelist, the Label Name parameter is required and the Label Value parameter is optional. If the LabelValue parameter is empty, containers whose Kubernetes labels contain the keys specified by LabelKey are matched. If the LabelValue parameter is not empty, containers whose Kubernetes labels consist of the key-value pairs specified by LabelKey and LabelValue are matched. By default, string matching is performed for the values of the LabelValue parameter. Containers are matched only if the values of the Kubernetes labels are the same as the values of the LabelValue parameter. If you specify a value that starts with a caret (`^`) and ends with a dollar sign (`$`), regular expression matching is performed. For example, if you set the LabelKey parameter to `app` and set the LabelValue parameter to `^(test1\|test2)$`, containers whose Kubernetes labels consist of app:test1 or app:test2 are matched. Key-value pairs are connected by using the OR operator. If a container has a Kubernetes label that consists of one of the specified key-value pairs, the container is matched. Note This parameter is available for Logtail V1.0.34 and later. If you change Kubernetes labels when Kubernetes control resources, such as Deployments, are running, the operational pod is not restarted. Therefore, the pod cannot detect the change. This may cause a matching rule to become invalid. When you configure the Kubernetes label whitelist and the Kubernetes label blacklist, we recommend that you use the Kubernetes labels of pods. For more information about Kubernetes labels, see Labels and Selectors. Kubernetes Pod Label Blacklist The Kubernetes label blacklist that specifies the containers from which stdout and stderr are not collected. If you configure a Kubernetes label blacklist, the Label Name parameter is required and the Label Value parameter is optional. If the LabelValue parameter is empty, containers whose Kubernetes labels contain the keys specified by LabelKey are filtered out. If the LabelValue parameter is not empty, containers whose Kubernetes labels consist of the key-value pairs specified by LabelKey and LabelValue are filtered out. By default, string matching is performed for the values of the LabelValue parameter. Containers are matched only if the values of the Kubernetes labels are the same as the values of the LabelValue parameter. If you specify a value that starts with a caret (`^`) and ends with a dollar sign (`$`), regular expression matching is performed. For example, if you set the LabelKey parameter to `app` and set the LabelValue parameter to `^(test1\|test2)$`, containers whose Kubernetes labels consist of app:test1 or app:test2 are matched. Key-value pairs are connected by using the OR operator. If a container has a Kubernetes label that consists of one of the specified key-value pairs, the container is filtered out. Note This parameter is available for Logtail V1.0.34 and later. If you change Kubernetes labels when Kubernetes control resources, such as Deployments, are running, the operational pod is not restarted. Therefore, the pod cannot detect the change. This may cause a matching rule to become invalid. When you configure the Kubernetes label whitelist and the Kubernetes label blacklist, we recommend that you use the Kubernetes labels of pods. For more information about Kubernetes labels, see Labels and Selectors.
Log Tag Enrichment	Specifies whether to add tags to logs. If you use Logtail V1.0.34 or later, we recommend that you add environment variables or Kubernetes labels to logs as log tags. Environment Variables The environment variables. After you configure this parameter, SLS adds environment variable-related fields to logs. For example, if you set the Environment Variable Name parameter to `VERSION` and the Tag Name parameter to `env_version`, SLS adds the __tag__:__env_version__: v1.0.0 field to logs if the environment variable settings of a container include `VERSION=v1.0.0`, Pod Labels The pod labels. After you configure this parameter, SLS adds Kubernetes pod label-related fields to logs. For example, if you set the Pod Label Name parameter to `app` and the Tag Name parameter to `k8s_pod_app`, SLS adds the __tag__:__k8s_pod_app__: serviceA field to logs if the label settings of a Kubernetes cluster include `app=serviceA`.
Maximum Log Length (Unit: Bytes)	The maximum size of a log. Default value: 524288. Valid values: 1024 to 20971520. If the size of a log exceeds the value of this parameter, Logtail stops matching the beginning of the first line of the next log and uploads the log to SLS.
Maximum Length of Traced Historical Data during First Collection (Unit: Bytes)	The maximum size of historical data that can be traced the first time Logtail collects logs from a log file. The default value is 131072 bytes, which is equivalent to 128 KB. Valid values: 131072 to 1048576.
Advanced Parameters	Specifies whether to manually configure specific parameters of a Logtail configuration. For more information, see CreateLogtailPipelineConfig.

Processor Configurations

Parameter	Description
Multi-line Mode	Regex to Match First Line The regular expression that is used to match the beginning of the first line of a log. If the beginning of a line matches the specified regular expression, the line is considered the first line of a new log. If the beginning of a line does not match the specified regular expression, the line is considered a part of the previous log. First Line Length Matched by Regex (Unit: Bytes) The size of the beginning of the first line of a log that matches the specified regular expression. Default value: 10240. Unit: bytes. You can configure this parameter to check whether the beginning of the first line of a log matches the specified regular expression. We recommend that you configure this parameter to improve match efficiency. First Line Match Timeout (Unit: Milliseconds) The timeout period for matching the beginning of the first line of a log based on the specified regular expression. Default value: 3000. If no new log is generated within 3,000 milliseconds, Logtail stops matching and uploads the last log to SLS. Note If the logs that you want to collect are multi-line logs, turn on Multi-line Mode.
Processing Method	You can select an extended processor from the Processor Type drop-down list in the Processor dialog box. For more information about processing plug-ins, see Overview of Logtail plug-ins for data processing.

Configure the parameters in the subsequent steps as prompted to create the Logtail configuration.

(Recommended) CRD - AliyunPipelineConfig

Warning

Only the Logtail component V0.5.1 or later supports AliyunPipelineConfig.

To create a Logtail configuration, you need to only create a custom resource(CR) by using the AliyunPipelineConfig custom resource definition (CRD). After you create the Logtail configuration, the system automatically applies the Logtail configuration.

Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
Run the following command to create a YAML file.
In the following command, cube.yaml is a sample file name. You can specify a different file name based on your business requirements.
```
vim cube.yaml
```

Enter the following script in the YAML file and specify the parameters based on your business requirements.

Important

You must configure a CR for each Logtail configuration. If multiple CRs are associated with the same Logtail configuration, the CRs other than the first CR do not take effect.
For more information about the parameters related to the AliyunPipelineConfig CRD, see (Recommended) Use AliyunPipelineConfig to manage a Logtail configuration. In this example, the Logtail configuration includes the basic settings used to collect container stdout and stderr. For more information about the parameters, see CreateLogtailPipelineConfig.
Make sure that the Logstore specified by the config.flushers.Logstore parameter exists. You can specify the spec.logstore parameter to automatically create a Logstore.

Collect stdout and stderr from specific containers in multi-line mode

In this example, a Logtail configuration named k8s-stdout- is created to collect stdout and stderr from the containers whose names contain app in multi-line mode. The collected logs are stored in a Logstore named k8s-stdout, which belongs to a project named k8s-log-clusterid.

apiVersion: telemetry.alibabacloud.com/v1alpha1
# Create a CR by using the ClusterAliyunPipelineConfig CRD.
kind: ClusterAliyunPipelineConfig
metadata:
  # Specify the name of the resource. The name must be unique within the Kubernetes cluster. The name is the same as that of the created Logtail configuration.
  name: example-k8s-stdout
spec:
  # Specify the project to which logs are collected.
  project:
    name: k8s-log-clusterid
  # Create a Logstore to store logs.
  logstores:
    - name: k8s-stdout
  # Configure the parameters for the Logtail configuration.
  config:
    # Optional. Specify the sample log.
    sample: |
      2024-06-19 16:35:00 INFO test log
      line-1
      line-2
      end
    # Configure the input plug-in.
    inputs:
      # Use the service_docker_stdout plug-in to collect stdout and stderr from containers.
      - Type: service_docker_stdout
        Stdout: true
        Stderr: true
        # Configure conditions to filter containers. Multiple conditions are evaluated by using a logical AND. 
        # Specify the namespace of the pod to which the required containers belong. Regular expression matching is supported. 
        K8sNamespaceRegex: "^(default)$"
        # Specify the name of the required containers. Regular expression matching is supported. 
        K8sContainerRegex: "^(.*app.*)$"
        # Configure settings for multi-line log collection.
        # Specify the regular expression that is used to match the beginning of the first line of a log.
        BeginLineRegex: \d+-\d+-\d+.*
    # Configure the output plug-in.
    flushers:
      # Use the flusher_sls plug-in to deliver logs to a specific Logstore. 
      - Type: flusher_sls
        Logstore: k8s-stdout
        Endpoint: cn-hangzhou.log.aliyuncs.com
        Region: cn-hangzhou
        TelemetryType: logs

Run the following command to apply the Logtail configuration. After you apply the Logtail configuration, Logtail starts to collect stdout and stderr from containers and sends the collected stdout and stderr to SLS.
In the following command, cube.yaml is a sample file name. You can specify a different file name based on your business requirements.
```
kubectl apply -f cube.yaml
```
Important
After logs are collected, you must create indexes before you can query and analyze the logs in the Logstore. For more information, see Create indexes.

CRD-AliyunLogConfig

To create a Logtail configuration, you need to only create a CR by using the AliyunLogConfig CRD. After the Logtail configuration is created, the system automatically applies the Logtail configuration.

Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
Run the following command to create a YAML file. In the following command, cube.yaml is a sample file name. You can specify a different file name based on your business requirements.
```
vim cube.yaml
```
Enter the following script in the YAML file and specify the parameters based on your business requirements.
Important
- The value of the configName parameter must be unique in the SLS project in which the Logtail component is installed.
- If multiple CRs are associated with the same Logtail configuration, the Logtail configuration is affected when you delete or modify one of the CRs. After a CR is deleted or modified, the status of the other associated CRs becomes inconsistent with the status of the Logtail configuration in SLS.
- For more information about CR parameters, see Use AliyunLogConfig to manage a Logtail configuration. In this example, the Logtail configuration includes the basic settings used to collect container stdout and stderr. For more information about the parameters, see CreateConfig.
Collect container stdout and stderr in single-line mode
In this example, a Logtail configuration named example-stdout-example is created to collect stdout and stderr from the containers whose names start with app in a Kubernetes cluster in single-line mode. The collected logs are stored in a Logstore named k8s-stdout, which belongs to a project named k8s-log-clusterid.
```
# The configurations for container stdout and stderr collection.
apiVersion: log.alibabacloud.com/v1alpha1
kind: AliyunLogConfig
metadata:
  # Specify the name of the resource. The name must be unique within the Kubernetes cluster. 
  name: simple-stdout-example
spec:
  # Specify the name of the Logstore. If the specified Logstore does not exist, SLS automatically creates a Logstore. 
  logstore: k8s-stdout
  # The Logtail configuration. 
  logtailConfig:
    # Specify the type of the data source. If you want to collect stdout and stderr, set the value to plugin. 
    inputType: plugin
    # Specify the name of the Logtail configuration. The name must be the same as the resource name that is specified in metadata.name. 
    configName: simple-stdout-example
    inputDetail:
      plugin:
        inputs:
          -
            # input type
            type: service_docker_stdout
            detail:
              # Specify that Logtail is used to collect both stdout and stderr. 
              Stdout: true
              Stderr: true
              K8sPodRegex: "^(app.*)$"
```
Run the following command to apply the Logtail configuration. After you apply the Logtail configuration, Logtail starts to collect stdout and stderr from containers and sends the collected stdout and stderr to SLS.
In the following command, cube.yaml is a sample file name. You can specify a different file name based on your business requirements.
```
kubectl apply -f cube.yaml
```
Important
After logs are collected, you must create indexes before you can query and analyze the logs in the Logstore. For more information, see Create indexes.

Preview container metadata

After you create a Logtail configuration, you can view container metadata and the reason why specific containers failed to match the filter conditions on the Logtail Configuration page.

Important

Only Logtail V1.4.0 and later that run on Linux or Logtail V1.4.0.0 and later that run on Windows support this feature.

In the Projects section, find the project that is used to create the Logtail configuration and click its name.
In the left-side navigation pane, click Log Storage. In the Logstores section, find the Logstore that is used to create the Logtail configuration, click the rightwards arrow> to the left of the Logstore name, and then choose Data Collection > Logtail Configurations.
On the Logtail Configuration page, find the Logtail configuration that you created and click its name.
On the Logtail Configuration page, click Edit.
Turn on Enable Container Metadata Preview.
Click Container Metadata Preview.
In the Container Preview dialog box, view container metadata.
- The Matched Containers tab displays the metadata of the containers that match the specified filter conditions.
- The All Containers tab displays the metadata of all containers in the current Kubernetes cluster and the reason why specific containers failed to match the filter conditions.

Log fields

The following table describes the fields that are uploaded by default for each log in a Kubernetes cluster.

Field	Description
_time_	The time when the log was collected.
_source_	The type of the log source. Valid values: stdout and stderr.
_image_name_	The name of the image.
_container_name_	The name of the container.
_pod_name_	The name of the pod.
_namespace_	The namespace of the pod.
_pod_uid_	The unique identifier of the pod.

Troubleshooting

If an exception occurs when you use Logtail to collect logs from containers, such as standard containers and Kubernetes containers, you can troubleshoot the issue based on the following references:

If the stderr logs cannot be parsed because the format of stderr logs is different from that of access logs, we recommend that you add indexes to the log content to facilitate querying for error-related log records. SLS does not allow you to use plug-ins to extract fields and parse logs based on the extracted fields. For more information, see Create indexes.