ApsaraDB for ClickHouse:Synchronize data from Log Service

Precautions

• ApsaraDB for ClickHouse does not support data of the Array type. For more information about data types that are supported by ApsaraDB for ClickHouse, see Data types.

• You are charged if you use Log Service. For more information, see Billing.

• Make sure that the Log Service project is deployed in the same region as your ApsaraDB for ClickHouse cluster.

Prerequisites

• Log Service is activated. For more information, see Activate Log Service.

• A project and a Logstore of Log Service are created in the region in which the ApsaraDB for ClickHouse cluster resides. Logs are collected and stored in the Logstore. For more information, see Quick Start.

• If you want to use a RAM user to ship log data from Log Service to ApsaraDB for ClickHouse, you must use your Alibaba Cloud account to grant the AliyunLogReadOnlyAccess permission to the RAM user. For more information, see Create a RAM user and authorize the RAM user to access Log Service.

• An ApsaraDB for ClickHouse table is created. The data structure and data type of the ApsaraDB for ClickHouse table map the data structure and data type of the Log Service data. For more information, see Create a table.

Procedure

1. Log on to the ApsaraDB for ClickHouse console.

2. On the Clusters page, find the cluster that you want to manage and click the cluster ID.

3. In the left-side navigation pane, click Log Delivery and then click Create Log Delivery Task.

4. On the Log Service Data Delivery page, configure the parameters. The following table describes the parameters.

   Parameter	Description
   Delivery Task Name	The name of the shipping task.
   Description	The description of the shipping task.
   Log Service Project	The name of the Log Service project.
   Logstore	The name of the Logstore.
   Cluster Name	The ID of the ApsaraDB for ClickHouse cluster.
   Database Name	The name of the ApsaraDB for ClickHouse database.
   Table Name	The name of the ApsaraDB for ClickHouse table.
   Account Name	The username of the database account.
   Account Password	The password of the database account.
   Field Mapping	The mappings between log fields and table fields of ApsaraDB for ClickHouse.
   Start Time	The point in time when logs are written to Log Service. For example, if you select 2021-12-31 08:00, logs that are written to Log Service after 2021-12-31 08:00 are shipped to ApsaraDB for ClickHouse.
   Whether to Filter Dirty Data	Specifies whether to filter out dirty data. We recommend that you turn on the switch. If you turn off the switch and dirty data that cannot be parsed exists, the shipping task fails. Note If you want to view information about dirty data of a shipping task, click Monitoring in the Actions column.

5. Click OK.

6. After you configure the parameters, you can view the shipping task on the Log Delivery page.

When the status of the shipping task changes from Pending to Synchronizing, log shipping is enabled.