After you enable monitoring for data transformation jobs, Simple Log Service sends alert notifications if exceptions occur during data transformation. This helps you identify exceptions at the earliest opportunity. This topic describes how to enable monitoring for data transformation jobs.
Prerequisites
A data transformation job is created. For more information, see Create a data transformation job.
Background information
After you create a data transformation job, Simple Log Service automatically creates a dashboard named Data Transformation Troubleshooting for the data transformation job. We recommend that you take note of the following metrics on the Data Transformation Troubleshooting dashboard:
System metrics: the data consumption latency of streaming data transformation and relevant exceptions.
Application metrics: the numbers of transformed logs and output logs.
For more information, see Data transformation dashboard.
Simple Log Service provides built-in alert resources for data transformation, including alert rule templates, action policies, and alert templates. You can use the resources based on the following rules:
You can use built-in alert rule templates to create alert rules without the need to write SQL statements. For example, you can use an alert rule created from a template to monitor the latency, exceptions, and failures of data transformation jobs.
You can specify notification methods and alert templates in built-in action policies.
You can specify the content of alert notifications in built-in alert templates.
Step 1: Configure an action policy
By default, built-in alert rule templates for data transformation are associated with built-in action policies for data transformation. Before you create an alert rule from a template, you must specify a notification method for the alert rule in the action policy.
Log on to the Simple Log Service console.
Go to the Action Policy tab.
In the Projects section, click the project that you want to manage.
In the left-side navigation pane, click Alerts.
On the Alert Center page, choose .
On the Action Policy tab, find the Simple Log Service built-in action policy for data transformation and click Edit in the Actions column.
In the Edit Action Policy dialog box, click the Primary Action Policy tab. On the Primary Action Policy tab, modify Notification Method to DingTalk-Custom and Request URL to the webhook URL of your DingTalk chatbot. Retain the default values of other parameters. Then, click Confirm.
For more information about how to obtain the webhook URL of a DingTalk chatbot, see DingTalk-Custom. You can configure other notification methods based on your business requirements. For more information, see Notification methods.
Step 2: Create an alert rule
Simple Log Service provides built-in alert rule templates. You can create an alert rule from a template based on your business requirements.
On the Alert Center page, click the Alert Rules tab. Then, click Create Alert. In the Create Alert panel, click Create from Template.
In the Create from Template panel, click SLS Data Transformation. Then, click the template that you want to use.
In the Create Alert panel, configure the parameters and click OK.
After you create the alert rule, Simple Log Service monitors all data transformation jobs of your project in real time.
If you want to monitor only a specific data transformation job, modify the query statement in the Query Statistics field in the Create Alert panel. On the Advanced Settings tab of the Query Statistics dialog box, specify the ID of the data transformation job that you want to monitor in the Query field.
Related operations
Operation | Description |
Create an alert rule | You can create an alert rule to monitor specific data transformation jobs. |
Disable an alert rule | If you disable an alert rule, the value in the Status column of the alert rule changes to Disabled, and alerts are no longer triggered based on the alert rule. The configurations of the alert rule are not deleted. If you want to enable the alert rule again, you do not need to reconfigure the parameters of the alert rule. |
Pause an alert rule | If you pause an alert rule, alerts are not triggered based on the alert rule within a specified period of time. |
Resume an alert rule | You can resume a paused alert rule based on your business requirements. |
Delete an alert rule | The configurations of the alert rule such as the specified IDs of data transformation jobs are deleted. If you want to use the alert rule to monitor jobs again, you must reconfigure the parameters of the alert rule. |
Reconfigure an alert rule | You can modify the parameters of an alert rule. For example, you can change the alert rule name, IDs of data transformation jobs that you want to monitor, monitoring thresholds, action policy, and alert severity. |