How do I troubleshoot the common errors that may occur when Simple Log Service collects logs? - Simple Log Service

This topic describes the common errors that may occur when Simple Log Service collects data and the solutions to the errors.

If you encounter other errors that are not described in this topic, submit a ticket.

Error	Description	Solution
LOG_GROUP_WAIT_TOO_LONG_ALARM	After a data packet is generated, the system waits for a long period of time to send the packet.	Check whether the system sends packets as expected, the data volume exceeds the default limit, the quota is insufficient, or network errors occur.
LOGFILE_PERMINSSION_ALARM	Logtail does not have the permissions to read the specified file.	Check whether the startup account of Logtail on the server is root. We recommend that you use the root account.
SPLIT_LOG_FAIL_ALARM	Logtail failed to split logs into lines because the regular expression that is specified to match the beginning of the first line of a log did not match the content of the logs.	Check whether the regular expression is correct. If you want to collect single-line logs, you can specify `.*` as the regular expression.
MULTI_CONFIG_MATCH_ALARM	By default, you can use only one Logtail configuration to collect logs from a log file. If you use multiple Logtail configurations to collect logs from a log file, only one Logtail configuration takes effect. Note You can use multiple Logtail configurations to collect the standard output (stdout) or standard errors (stderr) of Docker containers.	Delete redundant Logtail configurations. Modify configurations to allow Logtail to collect logs from a log file by using multiple Logtail configurations. For more information, see How do I collect multiple copies of logs in a file?.
REGEX_MATCH_ALARM	In full regex mode, the log content does not match the specified regular expression.	Copy the sample log in the error details to generate a new regular expression.
PARSE_LOG_FAIL_ALARM	In modes such as JSON and delimiter, Logtail failed to parse logs because the log format did not conform to the defined format.	Click the error link to view details.
CATEGORY_CONFIG_ALARM	The Logtail configuration is invalid.	In most cases, this issue occurs if the specified regular expression fails to extract a part of the file path as a topic. If this issue occurs due to other causes, submit a ticket.
LOGTAIL_CRASH_ALARM	Logtail stops responding because the resource usage of the server on which Logtail runs exceeds the upper limit.	Increase the upper limits of CPU utilization and memory usage. For more information, see Configure the startup parameters of Logtail.
REGISTER_INOTIFY_FAIL_ALARM	Logtail failed to register the log listener in Linux. This error may occur because Logtail does not have the permissions to access the folder on which Logtail listens or the folder has been deleted.	Check whether Logtail has the permissions to access the folder or the folder is deleted.
DISCARD_DATA_ALARM	The CPU resources that are configured for Logtail are insufficient, or throttling is triggered when network data is sent.	Increase the upper limits of CPU utilization or concurrent operations that can be performed to send network data. For more information, see Configure the startup parameters of Logtail.
SEND_DATA_FAIL_ALARM	No AccessKey pair is created for your Alibaba Cloud account. The server on which Logtail runs failed to connect to Simple Log Service, or the network quality is poor. The write quota on Simple Log Service is insufficient.	Create an AccessKey pair for your Alibaba Cloud account. Check the local configuration file /usr/local/ilogtail/ilogtail_config.json and run the curl <Server address> command to check whether content is returned. Increase the number of shards for the Logstore to allow more data to be written to the Logstore.
REGISTER_INOTIFY_FAIL_ALARM	Logtail failed to register the inotify watcher for the log directory.	Check whether the log directory exists and check the permission settings of the directory.
SEND_QUOTA_EXCEED_ALARM	The log write traffic exceeds the limit.	Increase the number of shards in the Simple Log Service console. For more information, see Split a shard.
READ_LOG_DELAY_ALARM	Log collection lags behind log generation. In most cases, this error occurs because the CPU resources that are configured for Logtail are insufficient or throttling is triggered when network data is sent.	Increase the upper limits of CPU utilization or concurrent operations that can be performed to send network data. For more information, see Configure the startup parameters of Logtail. When you import historical data, a large amount of data is collected in a short period of time. You can ignore this error.
DROP_LOG_ALARM	Log collection lags behind log generation, and the number of log files that are generated during rotation and are not parsed exceeds 20. In most cases, this error occurs because the CPU resources that are configured for Logtail are insufficient or throttling is triggered when network data is sent.	Increase the upper limits of CPU utilization or concurrent operations that can be performed to send network data. For more information, see Configure the startup parameters of Logtail.
LOGDIR_PERMINSSION_ALARM	Logtail does not have the permissions to read the log directory.	Check whether the log directory exists. If the log directory exists, check the permission settings of the directory.
ENCODING_CONVERT_ALARM	Encoding failed.	Check whether the configuration for log encoding is consistent with the actual implementation of log encoding.
OUTDATED_LOG_ALARM	The logs are expired. The log time lags behind the collection time for more than 12 hours. Possible causes: The log parsing progress lags behind the expected time for more than 12 hours. The custom time field is incorrectly configured. The time output of the log recording program is invalid.	Check whether the READ_LOG_DELAY_ALARM error is reported. If the error is reported, fix the error. If the error is not reported, check the configuration of the time field. Check the configuration of the time field. If the time field is correctly configured, check whether the time output of the log recording program is valid.
STAT_LIMIT_ALARM	The number of files in the log directory that is specified in the Logtail configuration exceeds the limit.	Check whether the log directory contains a large number of files and subdirectories. Reconfigure the log directory for monitoring and the maximum number of levels of subdirectories that you want to monitor. You can also modify the mem_usage_limit parameter. For more information, see Configure the startup parameters of Logtail.
DROP_DATA_ALARM	When the Logtail process exits, logs are dumped to the local disk. However, the dump operation times out. As a result, logs that are not dumped to the local disk are discarded.	In most cases, this error occurs because collection is severely blocked. Increase the upper limits of CPU utilization or concurrent operations that can be performed to send network data. For more information, see Configure the startup parameters of Logtail.
INPUT_COLLECT_ALARM	An error occurred when data was collected from the input data source.	Fix the error based on the error details.
HTTP_LOAD_ADDRESS_ALARM	The specified Addresses parameter in the Logtail configuration that is used to collect HTTP data is invalid.	Specify a valid value for the Addresses parameter.
HTTP_COLLECT_ALARM	An error occurred when HTTP data was collected.	Fix the error based on the error details. In most cases, this error is caused by timeout.
FILTER_INIT_ALARM	An error occurred when the filter was initialized.	In most cases, this error is caused by the invalid regular expressions of the filter. Fix the error based on the error details.
INPUT_CANAL_ALARM	An error occurred in the plug-in that is used to collect MySQL binary logs.	Fix the error based on the error details. When a Logtail configuration is updated, the canal service may restart. If the error is caused by the service restart, you can ignore the error.
CANAL_INVALID_ALARM	The plug-in that is used to collect MySQL binary logs is abnormal.	In most cases, this error is caused by inconsistent metadata. Metadata inconsistency may occur due to table scheme changes during running. Check whether table schemas are changed in the period during which the error is repeatedly reported. In other cases, submit a ticket.
MYSQL_INIT_ALARM	An error occurred when MySQL was initialized.	Fix the error based on the error details.
MYSQL_CHECKPOING_ALARM	The format of the checkpoints that are used to collect MySQL data is invalid.	Check whether to modify the checkpoint-related settings in the Logtail configuration. In other cases, submit a ticket.
MYSQL_TIMEOUT_ALARM	The MySQL query times out.	Check whether the MySQL server is properly connected to the network.
MYSQL_PARSE_ALARM	The MySQL query results failed to be parsed.	Check whether the format of the checkpoints that are used to collect MySQL data matches the format of the required fields.
AGGREGATOR_ADD_ALARM	The system failed to add data to the queue.	Data is sent too fast. If large amounts of data need to be sent, you can ignore this error.
ANCHOR_FIND_ALARM	An error occurred in the processor_anchor plug-in, an error occurred in the Logtail configuration, or logs that do not match the Logtail configuration exist.	Click the error link to view the sub-type of the error. The following sub-types are available. You can check the settings based on the error details of each sub-type. `anchor cannot find key`: The SourceKey parameter is configured in the Logtail configuration, but the specified fields are not found in logs. `anchor no start`: The system cannot find a match for the Start parameter in the value of SourceKey. `anchor no stop`: The system cannot find a match for the Stop parameter in the value of SourceKey.
ANCHOR_JSON_ALARM	An error occurred in the processor_anchor plug-in. The plug-in failed to expand the JSON data that is extracted based on the values of the Start and Stop parameters.	Click the error link to view details. View collected logs and related configurations to check whether configuration errors or invalid logs exist.
CANAL_RUNTIME_ALARM	An error occurred in the plug-in that is used to collect MySQL binary logs.	Click the error link to view details and perform troubleshooting based on the details. In most cases, this error is related to the primary ApsaraDB RDS for MySQL instance that is connected.
CHECKPOINT_INVALID_ALARM	The system failed to parse checkpoints.	Click the error link to view details and perform troubleshooting based on the details and the key-value pairs of the checkpoints in the details. The values of the checkpoints are indicated by the first 1,024 bytes in the checkpoint file.
DIR_EXCEED_LIMIT_ALARM	The number of directories on which Logtail listens at the same time exceeds the limit.	Check whether the Logtail configurations whose data is saved to the current Logstore and other Logtail configurations on the server on which Logtail is installed involve a large number of subdirectories. Reconfigure the log directory for monitoring and the maximum number of levels of subdirectories that you want to monitor for each Logtail configuration.
DOCKER_FILE_MAPPING_ALARM	The system failed to add a Docker file mapping by running a Logtail command.	Click the error link to view details and perform troubleshooting based on the details and the command in the details.
DOCKER_FILE_MATCH_ALARM	The specified file cannot be found in the Docker container.	Click the error link to view details and perform troubleshooting based on the container information and the file path that is used for the search.
DOCKER_REGEX_COMPILE_ALARM	An error occurred in the service_docker_stdout plug-in. The system failed to compile data based on the value of the BeginLineRegex parameter provided in the Logtail configuration.	Click the error link to view details and check whether the regular expression in the details is correct.
DOCKER_STDOUT_INIT_ALARM	The service_docker_stdout plug-in failed to be initialized.	Click the error link to view the sub-type of the error. The following sub-types are available: `host...version...error`: Check whether the Docker engine that is specified in the Logtail configuration can be accessed. `load checkpoint error`: The system failed to load the checkpoint file. If this error does not affect your business, you can ignore this error. `container...`: The specified container has invalid label values. Only stdout and stderr are supported. Perform troubleshooting based on the error details.
DOCKER_STDOUT_START_ALARM	The stdout size exceeds the limit when the service_docker_stdout plug-in is used to collect data.	In most cases, this error occurs because the stdout already exists the first time you use the plug-in. You can ignore this error.
DOCKER_STDOUT_STAT_ALARM	The service_docker_stdout plug-in cannot find the stdout.	In most cases, this error occurs because no stdout is available when a container is terminated. You can ignore this error.
FILE_READER_EXCEED_ALARM	The number of files that Logtail opens at the same time exceeds the limit.	In most cases, this error occurs because Logtail is collecting logs from a large number of files. Check whether the settings of the Logtail configuration are proper.
GEOIP_ALARM	An error occurred in the processor_geoip plug-in.	Click the error link to view the sub-type of the error. The following sub-types are available: `invalid ip...`: The system failed to obtain an IP address. Check whether the SourceKey parameter in the Logtail configuration is correctly configured or whether invalid logs exist. `parse ip...`: The system failed to parse an IP address into a city. Perform troubleshooting based on the error details. `cannot find key...`: The system cannot find a match for the SourceKey parameter in logs. Check whether the Logtail configuration is correct or whether invalid logs exist.
HTTP_INIT_ALARM	An error occurred in the metric_http plug-in. The plug-in failed to compile the regular expression that is specified by the ResponseStringMatch parameter in the Logtail configuration.	Click the error link to view details and check whether the regular expression in the details is correct.
HTTP_PARSE_ALARM	An error occurred in the metric_http plug-in. The plug-in failed to obtain HTTP responses.	Click the error link to view details and check the Logtail configuration or the requested HTTP server based on the details.
INIT_CHECKPOINT_ALARM	An error occurred in the plug-in that is used to collect binary logs. The plug-in failed to load the checkpoint file and started to process data from the beginning without a checkpoint.	Click the error link to view details and determine whether to ignore the error based on the details.
LOAD_LOCAL_EVENT_ALARM	Logtail performs local event handling.	In most cases, this error does not occur. If this error is caused by a non-human operation, you must perform troubleshooting. You can click the error link to view details and perform troubleshooting based on the file name, Logtail configuration name, project, and Logstore in the details.
LOG_REGEX_FIND_ALARM	Errors occur in the processor_split_log_regex and processor_split_log_string plug-ins. The plug-ins cannot find a match for the SplitKey parameter in logs.	Click the error link to view details and check whether configuration errors exist.
LUMBER_CONNECTION_ALARM	An error occurred in the service_lumberjack plug-in. The server was shut down while the plug-in was stopped.	Click the error link to view details and perform troubleshooting based on the details. In most cases, you can ignore this error.
LUMBER_LISTEN_ALARM	An error occurred in the service_lumberjack plug-in. The plug-in failed to perform listening during initialization.	Click the error link to view the sub-type of the error. The following sub-types are available: `init tls error...`: Check whether TLS-related configurations are correct based on the error details. `listen init error...`: Check whether address-related configurations are correct based on the error details.
LZ4_COMPRESS_FAIL_ALARM	An error occurred when Logtail performed LZ4 compression.	Click the error link to view details and perform troubleshooting based on the values of log lines, project, category, and region in the details.
MYSQL_CHECKPOINT_ALARM	An error occurred in the plug-in that is used to collect MySQL data. The error is related to checkpoints.	Click the error link to view the sub-type of the error. The following sub-types are available: `init checkpoint error...`: Checkpoint initialization failed. Check the checkpoint column specified by the Logtail configuration based on the error details and check whether the obtained values are correct. `not matched checkpoint...`: Logs do not match the checkpoint information. Check whether the mismatch is caused by user operations such as configuration update based on the error details. If the mismatch is caused by user operations, you can ignore the error.
NGINX_STATUS_COLLECT_ALARM	An error occurred in the nginx_status plug-in. The plug-in failed to obtain status information.	Click the error link to view details and perform troubleshooting based on the details and the URLs in the details.
NGINX_STATUS_INIT_ALARM	An error occurred in the nginx_status plug-in. The plug-in failed to initialize the URLs specified in parsing configurations.	Click the error link to view details and check whether the URLs in the details are correct.
OPEN_FILE_LIMIT_ALARM	Logtail failed to open the file because the number of opened files exceeded the limit.	Click the error link to view details and perform troubleshooting based on the file path, project, and Logstore in the details.
OPEN_LOGFILE_FAIL_ALARM	An error occurred when Logtail opened the file.	Click the error link to view details and perform troubleshooting based on the file path, project, and Logstore in the details.
PARSE_DOCKER_LINE_ALARM	An error occurred in the service_docker_stdout plug-in. The plug-in failed to parse the log.	Click the error link to view the sub-type of the error. The following sub-types are available: `parse docker line error: empty line`: The log is empty. `parse json docker line error...`: The system failed to parse the log into the JSON format. Perform troubleshooting based on the error details and the first 512 bytes of the log. `parse cri docker line error...`: The system failed to parse the log into the CRI format. Perform troubleshooting based on the error details and the first 512 bytes of the log.
PLUGIN_ALARM	An error occurred when plug-ins were initialized and called.	Click the error link to view the sub-type of the error. The following sub-types are available. You can perform troubleshooting based on the error details. `init plugin error...`: The system failed to initialize a plug-in. `hold on error...`: The system failed to suspend a plug-in. `resume error...`: The system failed to resume a plug-in. `start service error...`: The system failed to start a plug-in of the service input type. `stop service error...`: The system failed to stop a plug-in of the service input type.
PROCESSOR_INIT_ALARM	An error occurred in the processor_regex plug-in. The plug-in failed to compile the regular expression that is specified in the Logtail configuration.	Click the error link to view details and check whether the regular expression in the details is correct.
PROCESS_TOO_SLOW_ALARM	Logtail parses logs too slowly.	Click the error link to view details and check whether the parsing speed is acceptable based on the log quantity, buffer size, and parsing time in the details. If the speed is too slow, check whether other processes on the server on which Logtail is installed occupy excessive CPU resources or whether inappropriate parsing configurations exist, such as inefficient regular expressions.
REDIS_PARSE_ADDRESS_ALARM	An error occurred in the Redis plug-in. The plug-in failed to parse the value of the ServerUrls parameter that is provided in the Logtail configuration.	Click the error link to view details. Check the URLs for which the error is reported.
REGEX_FIND_ALARM	An error occurred in the processor_regex plug-in. The plug-in failed to find the fields that are specified by the SourceKey parameter in logs.	Click the error link to view details. Check whether the SourceKey parameter is correctly configured or whether the logs are valid.
REGEX_UNMATCHED_ALARM	An error occurred in the processor_regex plug-in. The match operation of the plug-in failed.	Click the error link to view the sub-type of the error. The following sub-types are available. You can perform troubleshooting based on the error details. `unmatch this log content...`: The system failed to match logs against the regular expression specified in the Logtail configuration. `match result count less...`: The number of matched fields is less than the number of fields specified by Keys in the Logtail configuration.
SAME_CONFIG_ALARM	Duplicate Logtail configurations are found for a Logstore. The most recent Logtail configuration that is found is discarded.	Click the error link to view details. Check whether configuration errors exist based on the details and the Logtail configuration path in the details.
SPLIT_FIND_ALARM	Errors occurred in the split_char and split_string plug-ins. The plug-ins failed to find the fields that are specified by the SourceKey parameter in logs.	Click the error link to view details. Check whether the SourceKey parameter is correctly configured or whether the logs are valid.
SPLIT_LOG_ALARM	Errors occurred in the processor_split_char and processor_split_string plug-ins. The number of parsed fields is different from the number of fields that are specified by the SplitKeys parameter.	Click the error link to view details. Check whether the SourceKey parameter is correctly configured or whether the logs are valid.
STAT_FILE_ALARM	An error occurred when the LogFileReader object was used to collect data from a file.	Click the error link to view details and perform troubleshooting based on the details and the file path in the details.
SERVICE_SYSLOG_INIT_ALARM	An error occurred in the service_syslog plug-in. The plug-in failed to be initialized.	Click the error link to view details. Check whether the Address parameter in the Logtail configuration is correctly configured.
SERVICE_SYSLOG_STREAM_ALARM	An error occurred in the service_syslog plug-in. The plug-in failed to collect data over TCP.	Click the error link to view the sub-type of the error. The following sub-types are available. You can perform troubleshooting based on the error details. `accept error...`: An error occurs when the Accept command is run. The plug-in waits for a while and tries again. `setKeepAlive error...`: The system failed to configure the Keep Alive parameter. The plug-in skips this error and continues. `connection i/o timeout...`: A TCP read times out. The plug-in modifies the timeout period and continues to read data. `scan error...`: A TCP read error occurs. The plug-in waits for a while and tries again.
SERVICE_SYSLOG_PACKET_ALARM	An error occurred in the service_syslog plug-in. The plug-in failed to collect data over UDP.	Click the error link to view the sub-type of the error. The following sub-types are available. You can perform troubleshooting based on the error details. `connection i/o timeout...`: A UDP read times out. The plug-in modifies the timeout period and continues to read data. `read from error...`: A UDP read error occurs. The plug-in waits for a while and tries again.
PARSE_TIME_FAIL_ALARM	The system failed to parse the log time.	You can use one of the following methods to identify the cause of the error and fix the error: Check whether the time field that is extracted by using a regular expression is correct. Check whether the value of the specified time field matches the time expression specified in the Logtail configuration.