This topic describes how to use Logtail to collect text logs from servers.
Prerequisites
A machine group has been created, and servers have been added to the machine group. We recommend that you create a custom identifier-based machine group. For more information, see Create a custom identifier-based machine group or Create an IP address-based machine group.
Ports 80 and 443 are enabled for the server on which Logtail is installed. If the server is an Elastic Computing Service (ECS) instance, you can reconfigure the related security group rules to enable the ports. For more information about how to configure a security group rule, see Add a security group rule.
The server from which you want to collect logs continuously generates logs. Logtail collects only incremental logs. If a log file on your server is not updated after a Logtail configuration is delivered and applied to the server, Logtail does not collect logs from the file. For more information, see Read log files.
Procedure
Log on to the Simple Log Service console.
In the Import Data section, search by Text Logs and click one of the cards that are displayed. In this example, click Multi-line - Text Logs.
In the Select Logstore step, select a project and a Logstore and click Next.
In the Machine Group Configurations step, configure a machine group.
Configure the Scenario and Installation Environment parameters based on your business requirements.
ImportantYou must configure the Scenario and Installation Environment parameters regardless of whether a machine group is available. The parameter settings affect subsequent configurations.
Make sure that a machine group is displayed in the Applied Server Groups section and click Next.
Machine group available
Select a machine group from the Source Machine Group section.
No machine group available
Click Create Machine Group. In the Create Machine Group panel, configure the parameters. You can set the Machine Group Identifier parameter to IP Address or Custom Identifier. For more information, see Create a custom identifier-based machine group or Create an IP address-based machine group.
ImportantIf you apply a machine group immediately after you create the machine group, the heartbeat status of the machine group may be FAIL. This issue occurs because the machine group is not connected to Simple Log Service. To resolve this issue, you can click Retry. If the issue persists, see What do I do if no heartbeat connections are detected on Logtail?
Create a Logtail configuration and click Next. A Logtail configuration requires up to 3 minutes to take effect.
Create indexes and preview data. Then, click Next. By default, full-text indexing is enabled in Simple Log Service. You can also manually create field indexes for the collected logs or click Automatic Index Generation. Then, Simple Log Service generates field indexes. For more information, see Create indexes.
ImportantIf you want to query all fields in logs, we recommend that you use full-text indexes. If you want to query only specific fields, we recommend that you use field indexes. This helps reduce index traffic. If you want to analyze fields, you must create field indexes. You must include a SELECT statement in your query statement for analysis.
Click Query Log. Then, you are redirected to the query and analysis page of your Logstore.
You must wait approximately 1 minute for the indexes to take effect. Then, you can view the collected logs on the Raw Logs tab. For more information about how to query and analyze logs, see Query and analyze logs.
References
If the preview page is blank or the query page displays no data when you use Logtail to collect logs, you can troubleshoot the issue by following the instructions provided in What do I do if errors occur when I use Logtail to collect logs? If you use Logtail to collect logs, errors may occur. For example, regular expressions may fail to be parsed, invalid file paths may exist, and traffic may exceed the processing capabilities of shards. You can view Logtail collection errors by following the instructions provided in How do I view Logtail collection errors? For more information about the common errors in data collection, see How do I troubleshoot the common errors that may occur when Simple Log Service collects logs?.
By default, you can use only one Logtail configuration to collect logs from a log file. If you want to collect multiple copies of logs in a file, you can follow the instructions provided in How do I collect multiple copies of logs in a file?
For more information about how to collect logs from servers in a corporate intranet to Simple Log Service, see Collect logs from servers in a corporate intranet.