After you create a data shipping job that ships data to Object Storage Service (OSS), you can monitor its performance, review its configuration, and manage its lifecycle from a single overview page. This page provides basic job information, a pre-built monitoring dashboard with real-time metrics, and options to modify or delete the job.
Prerequisites
Before you manage a data shipping job, make sure the following requirements are met:
A data shipping job is created to ship data to OSS. For more information, see Create an OSS data shipping job (new version).
The collection of job operational logs is enabled. For more information, see Enable the job operational log feature.
Open the Data Shipping Overview page
All management tasks -- viewing basic information, monitoring metrics, modifying settings, and deleting the job -- are performed on the Data Shipping Overview page. Follow these steps to open the page:
Log on to the Simple Log Service console.
In the Projects section, click the project that you want to manage.
In the left-side navigation pane, choose .
Click the OSS data shipping job that you want to manage. The Data Shipping Overview page appears.
View basic information
In the Basic Information section of the Data Shipping Overview page, you can view the following details about the data shipping job:
| Field | Description |
|---|---|
| Creation time | When the job was created. |
| Modification time | When the job was last modified. |
| Source Logstore | The Logstore from which the job reads data. |
| OSS bucket | The destination OSS bucket to which data is shipped. |
View the monitoring dashboard
When you create a data shipping job, Simple Log Service automatically creates a dashboard that displays real-time metrics for the job. You can use this dashboard to monitor data ingestion, delivery status, processing speed, and errors without any additional configuration.
The following table describes each chart on the dashboard.
Ingestion charts
| Chart name | Description |
|---|---|
| Ingest Success | The number of logs that are successfully read from the source Logstore. |
| Ingest Failed | The number of logs that failed to be read from the source Logstore. |
| Ingest PubNet (MB) | The traffic transmitted over the public network. This value is 0 because data transmission over the Internet is not supported. |
Delivery charts
| Chart name | Description |
|---|---|
| Deliver Success | The number of logs that are successfully shipped to OSS. |
| Deliver Failed | The number of logs that failed to be shipped to OSS. |
| Deliver PubNet (MB) | The traffic transmitted over the public network. This value is 0 because data transmission over the Internet is not supported. |
Processing charts
| Chart name | Description |
|---|---|
| Process Speed (events/s) | The number of logs that are processed per second. |
| Process Lags | The latency information of each shard in the Logstore. |
| Process Error | The exceptions that occur while the data shipping job is running. To view complete error logs, go to the internal-diagnostic_log Logstore in the project that you specified when you enabled the job operational log feature. Search for logs by using the keyword ERROR or WARN. For more information, see Enable the service log feature. |
| Process State | The status of the data shipping job, including the shipping time, number of shipped logs, number of failed logs, and average response time. |
Modify and restart a data shipping job
You can update the settings of a data shipping job at any time. After you save the changes, Simple Log Service automatically restarts the job.
You cannot change the name or start time of a data shipping job.
To modify a data shipping job, perform the following steps:
On the Data Shipping Overview page, click Modify LogShipper.
Modify the settings as needed.
Save the settings. Simple Log Service automatically restarts the job. The restart process requires a few seconds. You can click Refresh to check whether the restart is complete. After the job restarts, verify that the job is running and that data is being shipped to the specified OSS bucket as expected.
Delete a data shipping job
If you no longer need a data shipping job, you can delete it from the Data Shipping Overview page.
After you delete a data shipping job, the job cannot be restored. Proceed with caution.
To delete a data shipping job, choose in the upper-right corner of the Data Shipping Overview page.