Simple Log Service:Collect script execution logs

Prerequisites

Logtail 1.7.1 or later must be installed on your Linux server. For more information, see Install Logtail on a Linux server.

Usage notes

Logtail executes a configured script as a specified user to obtain script execution logs.

• The following script types are supported: bash, shell, python2, and python3.

• Specify a user to run the script to restrict the resources the script can access.

• If the script execution times out, the system automatically stops the script. Logtail does not collect the logs from this execution.

Procedure

1. Log on to the Simple Log Service console.

2. In the Import Data section, select Custom Data Plug-in.

3. Select the destination Project and Logstore, and then click Next.

4. Create a machine group.

   • To use an existing machine group, click Use Existing Machine Groups.

   • To create a machine group, perform the following steps. This example uses an ECS instance.

     1. On the ECS Instance tab, select your instance and click Create.

        For more information, see Install Logtail on ECS instances.

        Important

        If your server is an ECS instance that belongs to a different Alibaba Cloud account, a server from another cloud provider, or a self-managed server in a data center, you must manually install Logtail. For more information, see Install Logtail on a Linux server or Install Logtail on a Windows server. After you manually install Logtail, you must configure a user identifier on the server. For more information, see Configure a user identifier.

     2. Once the installation is complete, click Complete Installation.

     3. On the Create Machine Group page, enter a Name and click Next.

        SLS supports IP-based machine groups and custom identifier-based machine groups. For more information about the parameters, see Create an IP-based machine group and Create a custom identifier-based machine group.

5. Select and move the target machine group from the source machine groups to the applied machine groups, and then click Next.

   Important

   If you apply a machine group immediately after creating it, its heartbeat status may be FAIL because the connection has not yet been established. You can click Automatic Retry. If the issue persists, see Troubleshoot Logtail machine group issues in a host environment.

6. On the Configure Data Source tab, set the Configuration Name and Plug-in Configuration parameters, and then click Next.

   • inputs specifies the Logtail collection configuration. Required. Configure based on your data source.

     Important

     You can configure only one type of data source in an inputs block.

   • processors specifies the Logtail processing configuration. Optional. Configure one or more processing methods as needed.

     If the current inputs configuration does not meet your log parsing requirements, add a processors configuration in the Plug-in Config section to process the data. For example, extract fields, parse log time, mask data, or filter logs. For more information, see Use native and extension processing plug-ins.

   {
       "inputs":[
           {
               "type":"input_command",
               "detail":{
                   "ScriptType":"python2",
                   "User":"test",
                   "ScriptContent":"print(\"test input_command 0\")\nprint(\"test input_command 1\")\nprint(\"test input_command 2\")\nprint(\"test input_command 3\")\nprint(\"test input_command 4\")\nprint(\"test input_command 5\")\nprint(\"test input_command 6\")\nprint(\"test input_command 7\")\nprint(\"test input_command 8\")\nprint(\"test input_command 9\")\n",
                   "ContentEncoding":"PlainText",
                   "CmdPath":"/usr/bin/python",
                   "TimeoutMilliSeconds":1000,
                   "IgnoreError":false,
                   "Environments":[
                       "DEBUG=true"
                   ],
                   "IntervalMs":5000
               }
           }
       ]
   }

   Parameter	Type	Required	Description
   type	string	Yes	The data source type. Set to input_command.
   ScriptType	string	Yes	The script type. Supported types: bash, shell, python2, and python3.
   User	string	Yes	The username used to run the command. Only non-root users are supported. Note Ensure the specified username exists on the machine. Follow the principle of least privilege. For example, grant only rwx permissions on required directories or files.
   ScriptContent	string	Yes	The script content. The content can be plain text or Base64-encoded. Maximum length: 512 × 1024 bytes.
   ContentEncoding	string	No	The encoding format of the script content. Valid values: PlainText (default): The content is not encoded. Base64: The content is Base64-encoded.
   LineSplitSep	string	No	The separator for the script output. If empty, the output is not split and is returned as a single log entry.
   CmdPath	string	No	The path to the script interpreter. If empty, the default path is used: bash: /usr/bin/bash shell: /usr/bin/sh python2: /usr/bin/python2 python3: /usr/bin/python3
   TimeoutMilliSeconds	string	No	The timeout period for script execution. Unit: milliseconds. Default value: 3000.
   IgnoreError	Bool	No	Specifies whether to suppress error logs when the plugin fails. Default: false (errors are logged).
   Environments	[]string	No	The environment variables. Default: os.Environ(). Specified variables are appended to os.Environ().
   IntervalMs	int	Yes	The collection interval or script execution interval. Unit: milliseconds. Default value: 5000.

7. Preview the data, create an index, and then click Next.

   By default, SLS enables full-text indexing. Create field indexes manually based on the collected logs, or click Automatic Index Generation to have SLS create them automatically. For more information, see Create an index.

   Important

   To query and analyze logs, you must enable full-text indexing or field indexing. If both are enabled, field indexes take precedence.

8. Click Log Query. The query and analysis page for the Logstore opens.

   Wait for about one minute for the index to take effect. Then view the collected logs on the Raw Logs tab. For more information, see Get started with log query and analysis.

Example

Use a python2 script to retrieve the system time.

• Logtail collection configuration

  {
      "inputs":[
          {
              "type":"input_command",
              "detail":{
                  "ScriptType":"python2",
                  "User":"test",
                  "ScriptContent":"print(\"test input_command 0\")\nprint(\"test input_command 1\")\nprint(\"test input_command 2\")\nprint(\"test input_command 3\")\nprint(\"test input_command 4\")\nprint(\"test input_command 5\")\nprint(\"test input_command 6\")\nprint(\"test input_command 7\")\nprint(\"test input_command 8\")\nprint(\"test input_command 9\")\n",
                  "ContentEncoding":"PlainText",
                  "CmdPath":"/usr/bin/python",
                  "TimeoutMilliSeconds":1000,
                  "IgnoreError":false,
                  "Environments":[
                      "DEBUG=true"
                  ],
                  "IntervalMs":5000
              }
          }
      ]
  }

• Collected log

  content:"Mon Aug  7 02:17:52 UTC 2023"
  script_md5:"a9564ebc3289b7a14551baf8ad5ec60a"
  __pack_meta__:"1|MTY4OTI1MzExNTU2NTE2MzQ1NA==|2|1"
  __topic__:""
  __source__:"172.17.0.2"
  __tag__:__pack_id__:"D7B7F988051A3019-0"
  __tag__:__hostname__:"bc6c7472f257"
  __tag__:__client_ip__:"8.218.101.91"
  __tag__:__receive_time__:"1691374673"
  __time__:"1691374672"

• Field descriptions

  Field	Description
  content	The collected script execution log.
  script_md5	The MD5 hash of the script content.