How can I fix errors that occur during data outputs to the target Logstore? - Simple Log Service

This topic describes how to fix errors that may occur when data transformation results are written to the target Logstore.

Error causes and impact

After the log events in the source Logstore are transformed, the transformation results are exported to the target Logstore. The data transformation engine reads a batch of source log records (up to 10,000 log records) at a time. Transformed data is stored in a cache pool and then exported to the target Logstore after all source log records are transformed. Data output error

Errors that may occur during the data output to the target Logstore are due to the following causes:

The configurations of the target Logstore are incorrect.
The information of the target Logstore is changed.
A network connection error occurs.

Error impact:

If an error occurs when data is exported to the target Logstore, the transformation task will retry until the data export succeeds or is manually stopped. If the retry succeeds, the transformation task continues and no data loss or redundancy occurs.
If an error occurs after data is exported (for example, two target Logstores are specified. Data export from one Logstore succeeds, but data from the other Logstore fails), the transformation task saves the breakpoint and keeps retrying. After the retry succeeds, no data loss or redundancy occurs. If the transformation task is stopped and then restarted when the error occurs, the transformation task continues from the breakpoint. In this case, no data loss occurs, but data redundancy may occur.

Troubleshooting methods

The target Logstore is configured with an invalid AccessKey ID or AccessKey secret.

Error message

# The specified AccessKey ID is invalid.
{
  "errorCode": "Unauthorized", 
  "errorMessage": "AccessKeyId not found: LTAIL3gUus8A****"
}
#The specified AccessKey secret is invalid.
{ 
  "errorCode": "SignatureNotMatch", 
  "errorMessage": "signature uJfAJbc0ji04gb+cXhh0qWt****= not match"
}

Troubleshooting method
Check the data transformation rule to make sure that the specified AccessKey ID and AccessKey secret of the target Logstore are valid and correct.

The target project does not exist.
- Error message
```
{
  "errorCode": "ProjectNotExist", 
  "errorMessage": "The Project does not exist : your_project_name"
}
```
- Error cause
  The error message The Project does not exist may be reported due to the following causes:
  - The target project name specified in the data transformation rule is incorrect.
  - The specified target project name is correct, but the project was deleted during the transformation task.
- Troubleshooting method
  - Check whether the target project name specified in the data transformation rule is correct.
  - Check whether the target project is deleted.
The target project and Logstore do not exist.
Transformation rule
```
e_coutput("target1")
```
- Error message
```
{
  "errorMessage": "transform_data: output target target1 is not found in configurations"
}
```
- Error cause
  This error occurs because the target project and Logstore do not exist. The transformation rule exports log events to target1, but the error message target1 is not found in configurations is reported. It means the project and Logstore corresponding to target1 are not specified in the transformation rule.
- Troubleshooting method
  Make sure that the corresponding project and Logstore are specified in the transformation rule.
The information of the target Logstore is changed.
- Error cause
  This error occurred because the information of the specified target Logstore is changed during data transformation and the original configurations cannot be used to access the target Logstore.
- Error message
  The information of the target Logstore may have the following changes:
  - The target Logstore is deleted. In this case, the following error message is reported:
```
{
  "errorMessage": "Logstore [logstore_name] does not exist."
}
```
  - The AccessKey ID or AccessKey secret of the target Logstore is changed. In this case, the following error message is reported:
```
# The AccessKey ID is invalid.
{
  "errorCode": "Unauthorized", 
  "errorMessage": "AccessKeyId not found: LTAIL3gUus8A****"
}
# The AccessKey secret is invalid.
{ 
  "errorCode": "SignatureNotMatch", 
  "errorMessage": "signature uJfAJbc0ji04gb+cXhh0qWt****= not match"
}
```
- Troubleshooting method
  - Check whether the target Logstore is deleted.
  - Check whether the AccessKey ID or AccessKey secret of the target Logstore is changed.

A network connection error occurs.

Error message

{
  "errorCode": "LogRequestError",
  "errorMessage": "HTTPConnectionPool(host='your_host', port=80): Max retries exceeded with url: your_url (Caused by NewConnectionError: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'"
}

Troubleshooting method
Check whether the network connection is normal.