All Products
Search
Document Center

Simple Log Service:Collect text logs in NGINX configuration mode

Last Updated:Jan 22, 2025

NGINX logs record important information that can be used for website O&M. Simple Log Service allows you to collect NGINX logs in NGINX configuration mode and analyze the collected logs from multiple dimensions. You can create Logtail configurations to collect logs. This topic describes how to create a Logtail configuration in NGINX configuration mode in the Simple Log Service console.

Solution overview

In NGINX configuration mode, Logtail structures the log content based on the log format specified by the log_format parameter. Before you collect NGINX access logs, you must configure the log_format and access_log parameters in the /etc/nginx/nginx.conf file. The log_format parameter specifies the log format, and the access_log parameter specifies the storage path to the NGINX log files.

  • Log format and storage path

    In the following example, the default values of the log_format and access_log parameters are used:

    log_format main  '$remote_addr - $remote_user [$time_local] "$request" '
                     '$request_time $request_length '
                     '$status $body_bytes_sent "$http_referer" '
                     '"$http_user_agent"';
    
    access_log  /var/log/nginx/access.log  main;

    The following table describes the log fields.

    Field

    Description

    remote_addr

    The IP address of the client.

    remote_user

    The username that is used by the client to send the request.

    time_local

    The system time of the server. The value must be enclosed in brackets [].

    request

    The URI and the HTTP protocol of the request.

    request_time

    The time that is required to process the request. Unit: seconds.

    request_length

    The length of the request. The request line, request headers, and request body are all counted.

    status

    The status of the request.

    body_bytes_sent

    The number of bytes in the response that is sent to the client. The response header is not counted.

    http_referer

    The URL of the source web page.

    http_user_agent

    The browser information of the client.

  • Raw log

    Log that is generated by NGINX based on the log format specified by the log_format parameter:

    192.168.1.1 - - [11/Dec/2024:11:21:03 +0800] "GET /nginx-logo.png HTTP/1.1" 0.000 514 200 368 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
  • Log collected to the Simple Log Service Logstore:image

Prerequisites

  • A machine group is created, and servers are added to the machine group. For more information, see Create a custom identifier-based machine group and Create an IP address-based machine group.

  • Your server can connect to port 80 and port 443 of remote servers to ensure that Logtail can send logs to Simple Log Service.

  • The server from which you want to collect logs continuously generates logs. Logtail collects only incremental logs. If a log file on your server is not updated after a Logtail configuration is delivered and applied to the server, Logtail does not collect logs from the file. For more information, see Collection process.

Procedure

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

  3. On the Log Storage > Logstores tab, click the logstore that you want to manage.

  4. Click the drop-down arrow to the left of the Logstore, and then click Logtail Configurations. On the Logtail Configuration page, click Add Logtail Configuration.image

  5. In the Quick Data Import dialog box, choose Nginx - Text Logs > Integrate Now.image

  6. In the Machine Group Configurations step of the Import Data wizard, select the created machine group.image

  7. In the Logtail Configuration step of the Import Data wizard, configure the following parameters:

    1. Configuration Name: The name of the Logtail configuration. Example: nginx-logs.

    2. File Path: The storage path to the NGINX log files. For example, if you specify /var/log/nginx/**/access*, Simple Log Service collects logs from the NGINX log files whose names are prefixed with access in the /var/log/nginx directory and the recursive subdirectories of the directory.

    3. Processor Configurations: Click Data Parsing (NGINX Mode). In the Processor dialog box, configure the NGINX Log Configuration parameter. In most cases, the value of the NGINX Log Configuration parameter is prefixed with log_format. Simple Log Service automatically extracts fields. Example:

      log_format main  '$remote_addr - $remote_user [$time_local] "$request" '
                       '$request_time $request_length '
                       '$status $body_bytes_sent "$http_referer" '
                       '"$http_user_agent"';

      image

    4. Retain the default values for other parameters. For more information, see Collect text logs from servers.

  8. In the Query and Analysis Configurations step of the Import Data wizard, click Refresh to preview the collected logs.image

  9. Click Next to close the wizard. You can click Query Log. Then, you are navigated to the query and analysis page of your Logstore. You must wait approximately 1 minute for the indexes to take effect. Then, you can view the collected logs on the Raw Logs tab. For more information, see Guide to log query and analysis.

References