All Products
Search
Document Center

ApsaraDB for ClickHouse:Upgrade the major engine version by migrating data

Last Updated:Jul 09, 2024

An upgrade to the major engine version enhances performance, introduces new features, and fixes known bugs. This topic describes how to upgrade the major engine version by migrating data between ApsaraDB for ClickHouse Community-compatible Edition clusters.

Prerequisites

  • The source cluster and the destination cluster are ApsaraDB for ClickHouse Community-compatible Edition clusters.

  • The usernames and passwords of database accounts are created for the source ApsaraDB for ClickHouse cluster and the destination ApsaraDB for ClickHouse cluster.

  • The replicas of the source ApsaraDB for ClickHouse cluster and the destination ApsaraDB for ClickHouse cluster are consistent.

  • The source ApsaraDB for ClickHouse cluster and the destination ApsaraDB for ClickHouse cluster are deployed in the same region and use the same virtual private cloud (VPC).

  • If you enable tiered storage of hot data and cold data for the source cluster, you need to enable tiered storage of hot data and cold data for the destination cluster. If you disable tiered storage of hot data and cold data for the source cluster, you need to disable tiered storage of hot data and cold data for the destination cluster.

  • Each local table in the source ApsaraDB for ClickHouse cluster corresponds to a unique distributed table.

  • The available storage capacity of the disk for the destination ApsaraDB for ClickHouse cluster is greater than or equal to 1.2 times the used storage capacity of the disk for the source ApsaraDB for ClickHouse cluster.

  • The IP address of the source ApsaraDB for ClickHouse cluster is added to the IP address whitelist of the destination ApsaraDB for ClickHouse cluster.

  • The IP address of the destination ApsaraDB for ClickHouse cluster is added to the IP address whitelist of the source ApsaraDB for ClickHouse cluster.

Note

You can execute the select * from system.clusters; statement to query the IP address of an ApsaraDB for ClickHouse cluster. For more information about how to configure a whitelist, see Configure a whitelist.

Usage notes

  • You cannot downgrade the major engine version after the major engine version is upgraded.

  • Before you migrate data, make sure that no management operations, such as scale-out, upgrade, or downgrade operations, are being performed on the source ApsaraDB for ClickHouse cluster and the destination ApsaraDB for ClickHouse cluster.

  • Full data migration and incremental data migration are supported between ApsaraDB for ClickHouse clusters.

  • For non-MergeTree tables such as external tables and log tables, only the schemas are migrated. The data is not migrated. Therefore, if the source ApsaraDB for ClickHouse cluster contains non-MergeTree tables and you migrate data from the source ApsaraDB for ClickHouse cluster to the destination ApsaraDB for ClickHouse cluster, the non-MergeTree tables in the destination ApsaraDB for ClickHouse cluster have only a table schema and no specific business data.

  • After the major engine version is upgraded, the client fails to connect to the database service. You must change the endpoint for connecting the client to the database service.

Impacts of the major engine version upgrade

  • If you upgrade the major engine version for an ApsaraDB for ClickHouse cluster of version 20.8 or later by migrating data, you can read data from or write data to the source ApsaraDB for ClickHouse cluster during the data migration. To make sure that the migration task is complete as soon as possible, we recommend that you stop writing data to the source ApsaraDB for ClickHouse cluster when about 95% of the data has been migrated. If you upgrade the major engine version for an ApsaraDB for ClickHouse cluster of version 20.3 or earlier by migrating data, you can only read data from the source ApsaraDB for ClickHouse cluster during the data migration. After the data migration is complete, you can read data from or write data to the source ApsaraDB for ClickHouse cluster.

  • You cannot read data from or write data to the destination ApsaraDB for ClickHouse cluster until the data migration is complete.

Supported versions

The following table describes the supported upgrades for each major engine version of the source ApsaraDB for ClickHouse cluster.

Engine version before the upgrade

Engine version after the upgrade

V19.15

V21.8 or V22.8

V20.3

V21.8 or V22.8

V20.8

V21.8 or V22.8

V21.8

V22.8

Step 1: Create a destination ApsaraDB for ClickHouse cluster

Create an ApsaraDB for ClickHouse Community-compatible Edition cluster of a later version. For more information about the supported versions and the procedure for creating an ApsaraDB for ClickHouse Community-compatible Edition cluster, see the Supported versions section of this topic and Create an ApsaraDB for ClickHouse cluster.

Step 2: Migrate data to the destination ApsaraDB for ClickHouse cluster

Important

In this step, you must log on to the destination cluster, instead of the source cluster.

  1. Log on to the ApsaraDB for ClickHouse console.

  2. On the Clusters page, click the Default Instances tab, and then click the ID of the cluster that you want to manage.

  3. In the left-side navigation pane of the page that appears, click Migrate Instance.

  4. On the page that appears, click Create Migration Task.

    实例迁移

    Note

    If no data is migrated to the selected cluster, the message No data found. is displayed.

  5. In the Select the source and destination instances step, configure Instance ID, Database Account, and Database Password in the Source Instance Information section. Then, configure Database Account and Database Password in the Destination Instance Information section.

    选择源及目标实例

  6. Click Test Connectivity and Proceed to test the connectivity between the two clusters.

    • If the connection is established, proceed to the next step.

    • If the connection fails to be established, check the error message. If the message Source instance account or password error. appears, enter a valid value for the Database Account parameter or the Database Password parameter.

  7. After the source cluster and destination cluster are connected, proceed to the Migration Content step. Click Next: Pre-detect and Start Synchronization.

    迁移内容配置

    Note

    You can migrate the following objects from the source cluster: the cluster, databases, tables, data dictionaries, materialized views, user permissions, and cluster configurations.

  8. After the migration task is configured, the system pre-checks the migration configuration and starts the migration task in the backend.

    • If no issues are detected during the precheck, click Completed.

    • If an issue is detected, follow the instructions to resolve the issue and migrate the data again. The following table describes the precheck items.

      Item

      Description

      Instance Status Detection

      Before you migrate data, make sure that no management operations, such as scale-out, upgrade, or downgrade operations, are being performed on the source ApsaraDB for ClickHouse cluster and the destination ApsaraDB for ClickHouse cluster. If management operations are being performed on the source cluster and the destination cluster, the system cannot start a migration task.

      Storage Space Detection

      Before the system starts a migration task, the system checks the storage space of the source cluster and the destination cluster. Make sure that the available storage capacity of the disk for the destination cluster is greater than or equal to 1.2 times the used storage capacity of the disk for the source cluster.

      Local Table and Distributed Table Detection

      If no distributed table is created for a local table or multiple distributed tables are created for the same local table, the precheck fails. You must delete redundant distributed tables to ensure that the distributed table of each local table is unique.

  9. After the migration task is complete, you can view the migration task on the Migrate Instance page.

    迁移任务管理

    Note

    If Completed is displayed in the Migration Status column, data is migrated from the source cluster to the destination cluster.

Step 3: Delete the source ApsaraDB for ClickHouse cluster

For more information about how to delete a source cluster, see Delete a cluster.

Warning

After the source cluster is deleted, all data in the source cluster is deleted and cannot be restored. Before you delete the source cluster, make sure that all business data in the source cluster is migrated to the destination cluster.