Promo Center

50% off for new user

Direct Mail-46% off

Learn More
This topic was translated by AI and is currently in queue for revision by our editors. Alibaba Cloud does not guarantee the accuracy of AI-translated content. Request expedited revision

How do I troubleshoot an error that is related to a Logtail machine group in a host environment?

Updated at: 2025-01-08 16:25

This topic describes how to address the issue of a Logtail machine group not having heartbeats in a host environment.

Machine group installation examples

Troubleshooting overview

  1. Step 1: Verify Logtail is running as expected: Ensure Logtail is operational on the server.

  2. Step 2: Confirm the IP address in the machine group matches the IP address retrieved by Logtail: Ensure the IP address listed in the machine group matches the IP address in Logtail's app_info.json file. Discrepancies will cause heartbeat failures.

  3. Step 3: Verify the Validity of Logtail Startup Parameters: Ensure the project region specified in the ilogtail_config.json file is correct.

  4. Step 4: Verify network connectivity: Check the server's network connection to the project.

  5. Step 5: Ensure the server's system time is accurate: Adjust the system time if it deviates from the local time.

  6. Step 6: Confirm the presence of a user identifier: If the project and ECS server are under different accounts or with other clouds or self-managed servers, verify the user identifier.

  7. Step 7: Check for a configured custom identifier: For custom identifier-based machine groups, ensure a custom identifier is set up.

  8. Step 8: Restart Logtail: After completing the above steps, restart Logtail.

What to do next

How to view Logtail error messages: If heartbeats are restored but logs are not being collected, continue troubleshooting by examining Logtail error messages.

Step 1: Check whether Logtail is running as expected

Linux system

  1. Log on to the server where Logtail is installed.

  2. Execute the following command:

    ps -ef | grep ilogtail
    • If two records appear, Logtail is operating correctly. These records represent the Logtail daemon and worker processes.

      UID          PID    PPID  C STIME TTY          TIME CMD
      ...
      root          12       1  0 Nov10 ?        00:00:00 /usr/local/ilogtail/ilogtail
      root          14      12  0 Nov10 ?        03:07:43 /usr/local/ilogtail/ilogtail
      ...
      Important

      If more than two Logtail processes are found, this indicates multiple instances are active on the server, which could lead to duplicate data collection. Ensure this aligns with your requirements.

    • If no Logtail processes are detected:

      • Logtail is installed but not running. For more information, see how to start and stop Logtail (Linux System).

      • Logtail is not installed. Proceed with the installation. For guidance, see Install Logtail (Linux System).

        Important

        During installation, ensure you choose an operating system compatible with Logtail installation. Set the installation parameters according to the Simple Log Service project's region and choose the appropriate installation method for your network type. For details on network types, refer to selecting a network.

Windows system

  1. Log on to the server where Logtail is installed.

  2. Open the Run window and type services.msc.

  3. Check the status of the LogtailDaemon service (Logtail V1.0.0.0 or later) or the LogtailWorker service (Logtail V0.x.x.x).

    If these services are not active:

    Important

    During installation, ensure you select an operating system compatible with Logtail. Choose installation parameters according to the region associated with the Simple Log Service project, and select the appropriate installation method for your network type. For more information on network types, see selecting a network.

Step 2: Check whether the IP address that is specified in the machine group is the same as the IP address that is obtained by Logtail

Note

Logtail retrieves the IP address of a Linux server using the following methods:

  • If the server's hostname is not associated with an IP address, Logtail retrieves the IP address of the server's first network interface card (NIC).

  • To specify a custom IP address, set the working_ip parameter in the ilogtail_config.json file as detailed in Step 3. Once set, the ip field in the app_info.json file will automatically update to the working_ip value. For more information about working_ip, see Set startup parameters.

  • If the server's hostname is mapped to an IP address in the /etc/hosts file, Logtail uses the corresponding IP address.

  1. Retrieve the ip field value from the app_info.json file.

    The default file path varies by operating system, as outlined in the table below:

    Operating system

    Logtail

    Path of the app_info.json file

    Operating system

    Logtail

    Path of the app_info.json file

    Linux

    Logtail for 64-bit Linux

    /usr/local/ilogtail/app_info.json

    64-bit Windows

    Logtail for 64-bit Windows

    C:\Program Files\Alibaba\Logtail\app_info.json

    Logtail for 32-bit Windows

    C:\Program Files (x86)\Alibaba\Logtail\app_info.json

    32-bit Windows

    Logtail for 32-bit Windows

    C:\Program Files\Alibaba\Logtail\app_info.json

    Logtail records the retrieved IP address in the app_info.json file's ip field.

    {
      "UUID" : "",
      "hostname" : "iZ8vbdlzf******azuhZ",
      "instance_id" : "E9633380-***********-00163E1AA597_172.16.2.200_166****11",
      "ip" : "172.**.**.200",
      "logtail_version" : "1.3.1",
      "os" : "Linux; 4.19.91-26.1.al7.x86_64; #1 SMP Tue Jul 26 17:52:28 CST 2022; x86_64",
      "update_time" : "2022-12-27 05:38:33"
    }
  2. Verify that the IP address retrieved by Logtail is listed in the machine group.

    Simple Log Service machine groups are either IP address-based or custom identifier-based. For more information, see Machine group.

    • IP address-based machine group: Confirm that the IP Address includes the IP address retrieved in the previous step.

      If the IP Address text box contains other destination Logtail IP addresses, such as a public IP, update it to the IP address retrieved by Logtail and check for a normal heartbeat. If the heartbeat is normal, troubleshooting can cease.

    • Custom identifier-based machine group: Confirm that the Machine Group Status reflects the IP address retrieved in the previous step. If the Heartbeat indicates OK, you can conclude the troubleshooting process.image..png

Step 3: Check whether the Logtail startup parameters are valid

The ilogtail_config.json file contains the startup parameters for Logtail.

  1. Log on to the server where Logtail is installed.

  2. Locate the ilogtail_config.json file.

    The default location of the file depends on the operating system, as detailed in the table below:

    Operating system

    Logtail

    Path of the ilogtail_config.json file

    Operating system

    Logtail

    Path of the ilogtail_config.json file

    Linux

    Logtail for 64-bit Linux

    /usr/local/ilogtail/ilogtail_config.json

    64-bit Windows

    Logtail for 64-bit Windows

    \ilogtail_config.json

    Logtail for 32-bit Windows

    C:\Program Files (x86)\Alibaba\Logtail\ilogtail_config.json

    32-bit Windows

    Logtail for 32-bit Windows

    C:\Program Files\Alibaba\Logtail\ilogtail_config.json

    1. Open the ilogtail_config.json file to verify the validity of the parameters within.

      {
        "config_server_address" : "http://logtail.<config_region>.log.aliyuncs.com",
        "data_server_list" :
        [
          {
            "cluster" : "<project region>",
            "endpoint" : "<endpoint>"
          }
        ],
        ...
      }
      • The startup parameters in the ilogtail_config.json file are considered valid if they match the descriptions in the table below.

      • Should the Logtail startup parameters be invalid, adjust the ilogtail_config.json file as described in the table below and restart Logtail. For more information, see Restart Logtail.

        To determine the service region for a specific project, see Service region.

        Scenario

        Network type

        <config_region>

        <endpoint>

        The server is an Elastic Compute Service (ECS) instance that resides in the same region as the project

        Alibaba Cloud internal network

        <project region>-intranet

        <project region>-intranet.log.aliyuncs.com

        Other scenarios

        Public network

        <project region>

        <project region>.log.aliyuncs.com

        Transfer acceleration

        log-global.aliyuncs.com

Step 4: Check whether network connectivity is available

Logtail can only upload data to Simple Log Service if it can establish a connection to the following addresses from the server it operates on.

Important

When using the internal network, you must append -intranet after <endpoint>.

  1. The address specified by the config_server_address field in the ilogtail_config.json file, along with its HTTPS counterpart.

  2. http://<project name>.<endpoint>.

    • You can find the project name and region as follows.

      image

    • The <endpoint> is the address specified by the data_server_list.endpoint field in the ilogtail_config.json file.

  3. http://ali-<project region>-sls-admin.<endpoint>. The <endpoint> is the address specified by the data_server_list.endpoint field in the ilogtail_config.json file.

To verify the network connection, follow these steps:

Linux system

  1. Log on to the server where Logtail is installed.

  2. Execute the curl command to connect to the listed addresses in order.

    curl http://<project name>.cn-hangzhou-intranet.log.aliyuncs.com

    If the output resembles the following example, the network is connected.

    {"Error":{"Code":"OLSInvalidMethod","Message":"The script name is invalid : /","RequestId":"5D****09"}}

    If the network is not connected, ensure that ports 80 and 443 are open, the destination address is not blocked, and other network configurations, such as DNS settings and security groups, are correct.

Windows system

  1. Log on to the server where Logtail is installed.

  2. Execute the telnet command to connect to the listed addresses in order.

    telnet <project name>.cn-hangzhou-intranet.log.aliyuncs.com 80 # If you use the HTTPS protocol, the port number is 443.

    If the output resembles the following example, the network is connected.

    Trying 100*0*7*5...
    Connected to xxx.
    Escape character is '^]'.

    If the network is not connected, ensure that ports 80 and 443 are open, the destination address is not blocked, and other network configurations, such as DNS settings and security groups, are correct.

Step 5: Check whether the system time of the server on which Logtail is installed is correct

Linux system

  1. Log on to the server where Logtail is installed.

  2. Execute the date command to check the system time.

    Wed Dec 28 06:59:26 UTC 2022

    If the system time does not match the local time, take the following steps:

    • Adjust the system time to match the local time.

    • If adjusting the system time is not possible, insert the "enable_log_time_auto_adjust": true configuration into the ilogtail_config.json file. Modify the file and then restart Logtail. For more information, see Restart Logtail. For details on the location of the ilogtail_config.json file, refer to Step 3: Verify the validity of Logtail startup parameters.

Windows system

  1. Log on to the server where Logtail is installed.

  2. Check the time displayed on the taskbar at the bottom right of the desktop.

    • Set the system time to match the local time zone.

    • If you are unable to adjust the system time, insert the parameter "enable_log_time_auto_adjust": true into the ilogtail_config.json file. Modify the file and then restart Logtail. For more information, see Restart Logtail. For details on the ilogtail_config.json file location, see Step 3: Verify the Logtail startup parameters.

Step 6: Check whether a user identifier is specified

Important
  • If you are utilizing other cloud services, self-managed servers, or if the project and ECS server are under different accounts, it is crucial to verify the user identifier.

  • The user identifier must correspond to the ID of an Alibaba Cloud account. For more information, see how to configure a user identifier.

You can check for the presence of a user identifier by examining the user identifier file in a specific directory. If the result is empty, ensure that a user identifier file is present in the designated path.

Note

The location of the user identifier file depends on the operating system:

  • Linux: /etc/ilogtail/users/

  • Windows: C:\LogtailData\users\

  • If the specified directory lacks a user identifier file or contains an invalid one, follow these steps to resolve the issue:

    • For Linux: Execute the command cd /etc/ilogtail/users/ && touch <uid> to create a user identifier file, where <uid> is the Alibaba Cloud account ID associated with the project.

    • For Windows: Navigate to the directory C:\LogtailData\users\ and create a blank file named <uid>, where <uid> is the Alibaba Cloud account ID associated with the project.

  • A valid user identifier file is one that is named after the Alibaba Cloud account ID to which the current project is linked.

Step 7: Check whether a custom identifier is configured if you use a custom identifier-based machine group

You can check if a custom identifier is set up on the server by examining the user_defined_id file in the designated directory.

  • If the output is empty, verify the existence of the user_defined_id file or whether a custom identifier is present within it.

    Note

    The location of the user_defined_id file depends on the operating system:

    • Linux: /etc/ilogtail/user_defined_id

    • Windows: C:\LogtailData\user_defined_id

    • If the user_defined_id file is missing, create one with the name 'user_defined_id' and include the machine group's custom identifier. For more information, see Configure a custom identifier.

    • If the user_defined_id file lacks a custom identifier or contains an invalid one, insert a new line with the correct custom identifier for the machine group. For more information, see Configure a custom identifier.

    • If the user_defined_id file already includes the specified custom identifier for the machine group, then it is correctly configured.

Step 8: Restart Logtail

After completing the previous steps, you must restart Logtail.

Linux system

  1. Log on to the Linux server where Logtail is installed.

  2. Execute the following command:

    sudo /etc/init.d/ilogtaild restart

Windows system

  1. Log on to the Windows server where Logtail is installed.

  2. Open the Run window and type services.msc.

  3. Restart the LogtailDaemon service (for Logtail V1.0.0.0 or later) or the LogtailWorker service (for Logtail V0.x.x.x).

What to do next

If the issue remains unresolved after you have carried out the suggested steps, please submit a or contact support.

  • On this page (1)
  • Machine group installation examples
  • Troubleshooting overview
  • What to do next
  • Step 1: Check whether Logtail is running as expected
  • Linux system
  • Windows system
  • Step 2: Check whether the IP address that is specified in the machine group is the same as the IP address that is obtained by Logtail
  • Step 3: Check whether the Logtail startup parameters are valid
  • Step 4: Check whether network connectivity is available
  • Linux system
  • Windows system
  • Step 5: Check whether the system time of the server on which Logtail is installed is correct
  • Linux system
  • Windows system
  • Step 6: Check whether a user identifier is specified
  • Step 7: Check whether a custom identifier is configured if you use a custom identifier-based machine group
  • Step 8: Restart Logtail
  • Linux system
  • Windows system
  • What to do next
Feedback
phone Contact Us

Chat now with Alibaba Cloud Customer Service to assist you in finding the right products and services to meet your needs.

alicare alicarealicarealicare