All Products
Search

This Product

    Tair (Redis® OSS-Compatible):Verify migrated Redis data

    Document Center

    Tair (Redis® OSS-Compatible):Verify migrated Redis data

    Last Updated:Apr 10, 2024

    After Redis data is migrated, you can use the redis-full-check tool to check whether the data is consistent between the source and destination instances.

    Prerequisites

    • Redis data is migrated.

    • An Elastic Compute Service (ECS) Linux instance is created for running the redis-full-check tool. For more information, see Create an ECS instance.

    • The ECS instance can access the source and destination Redis instances.

      Note

      • If the ECS instance and the source or destination Redis instance are in the same virtual private cloud (VPC), you must add the internal IP address of the ECS instance to a whitelist of the Redis instance. For more information, see Configure whitelists.

      • If the ECS instance and the source or destination Redis instance are not in the same VPC, the ECS instance can access the Redis instance over the public endpoint. For more information, see Use a public endpoint to connect to an ApsaraDB for Redis instance.

    Scenario

    If an exception occurs during Redis data migration, the data is inconsistent between the source and destination Redis instances. You can use the redis-full-check tool to locate abnormal data, which provides a reliable basis for data alignment.

    redis-full-check is a Redis data verification tool developed in-house by Alibaba Cloud. It can extract data from the source and destination instances, conduct multiple rounds of comparisons of the extracted data, and then record the comparison results in an SQLite3 database. This tool can be used to verify full data.

    Note

    For more information about the redis-full-check tool, see redis-full-check on GitHub.

    Procedure

    1. Log on to the ECS instance. For more information, see Connect to an instance.

    2. Download the redis-full-check tool on the ECS instance.

      Note

      We recommend that you download the latest version of redis-full-check.

    3. Run the following command to decompress the redis-full-check.tar.gz package: 

      tar -xvf redis-full-check.tar.gz

    4. Run the following command to verify data: 

      ./redis-full-check -s "<Endpoint 1 of the source Redis instance:Port number of endpoint 1;Endpoint 2 of the source Redis instance:Port number of endpoint 2;Endpoint 3 of the source Redis instance:Port number of endpoint 3;>" -p <Password of the source Redis instance> -t <Endpoint of the destination Redis instance:Port number> -a <Password of the destination Redis instance> --m=1 --sourcedbfilterlist=0;2 --targetdbfilterlist=0;2

      Sample command:

      ./redis-full-check -s "10.xx.xx.1:6379" -p SourcePwd233 -t "r-bp1zc********.redis.rds.aliyuncs.com:6379" -a testaccount:Test1234 --m=1 --sourcedbfilterlist=0;2 --targetdbfilterlist=0;2

      The following table describes common parameters of the redis-full-check tool. For more information, see Configure redis-full-check.

      Table 1. Common parameters of redis-full-check

      Parameter

      Description

      Example

      -s

      The endpoint and port of the source Redis instance.

      Note

      • If the source Redis instance is a cluster, separate cluster endpoints with semicolons (;).

      • Enclose the cluster endpoints in a pair of double quotation marks (").

      • This parameter is required. 

      r-bp1xxxxxxxxxxxxx.redis.rds.aliyuncs.com:6379
      "10.xx.xx.1:7000;10.xx.xx.1:7001;10.xx.xx.2:7002;10.xx.xx.2:7003"

      -p

      The username and password that are used to connect to the source Redis instance. Specify this parameter in the <username>:<password> format. Example: testaccount:Test1234. If no username is set for the instance, you can enter only the password. If no password is set for the instance, you can leave the parameter empty.

      SourcePwd233 or testaccount:Test1234

      -t

      The endpoint and port of the destination Redis instance.

      Note

      • If the destination Redis instance is a cluster, separate cluster endpoints with semicolons (;).

      • Enclose the cluster endpoints in a pair of double quotation marks (").

      • This parameter is required. 

      r-bp1xxxxxxxxxxxxx.redis.rds.aliyuncs.com:6379
      "10.xx.xx.1:7000;10.xx.xx.1:7001;10.xx.xx.2:7002;10.xx.xx.2:7003"

      -a

      The username and password that are used to connect to the destination Redis instance. Specify this parameter in the <username>:<password> format. Example: testaccount:Test1234. If no username is set for the instance, you can enter only the password. If no password is set for the instance, you can leave the parameter empty.

      TargetPwd233 or testaccount:Test1234

      --sourcedbtype

      The type of the source Redis instance. Default value: 0. Valid values:

      • 0: standalone or standard

      • 1: cluster (direct connection mode)

      • 2: cluster (proxy mode)

      --sourcedbtype=1

      --sourcedbfilterlist

      The databases for which you want to verify data in the source Redis instance.

      Note

      • This parameter is not required for open source Redis clusters.

      • If you do not set this parameter, the data in all databases is verified.

      • Separate multiple databases with semicolons (;).

      --sourcedbfilterlist=0;1;2

      --targetdbtype

      The type of the destination Redis instance. Valid values:

      • 0: standalone or standard

      • 1: cluster (direct connection mode)

      • 2: cluster (proxy mode)

      --targetdbtype=0

      --targetdbfilterlist

      The databases for which you want to verify data in the destination Redis instance.

      Note

      • This parameter is not required for open source Redis clusters.

      • If you do not set this parameter, the data in all databases is verified.

      • Separate multiple databases with semicolons (;).

      --targetdbfilterlist=0;1;2

      -d

      The name of the file for storing inconsistent keys. Default value: result.db.

      xxx.db

      --comparetimes

      The number of verifications. Default value: 3. Minimum value: 1. We recommend that you set this parameter to a value that does not exceed 5.

      --comparetimes=1

      -m

      The verification mode. Default value: 2. Valid values:

      • 1: verifies full data.

      • 2: only verifies the length of the value.

      • 3: only checks whether the keys exist.

      • 4: verifies full data except for large keys.

      -m=1

      --qps

      The queries per second (QPS). Default value: 15000.

      Note

      • Minimum value: 1.

      • Maximum value: depends on the server performance.

      --qps=10

      --filterlist

      The keys to verify. Separate multiple keys with vertical bars (|).

      Note

      • abc*: matches all keys that start with abc.

      • abc: matches the abc key.

      --filterlist=abc*|efg|m*

      Note

      After the verification is completed, the comparison result is displayed on the CLI. The following result indicates that two keys are inconsistent between the source and destination Redis instances. If the number of inconsistent keys is 0, the data is consistent between the source and destination Redis instances. 

      all finish successfully, totally 2 keys or fields conflict

    5. Check the SQLite3 database that stores the inconsistent keys.

      1. Run the sqlite3 result.db.3 command.

        Note

        Inconsistent keys are stored in result.db.3 by default.

      2. Execute the SELECT * FROM key; statement.

        Figure 1. View the list of inconsistent keys

        Note

        The SQLite3 database contains the key and field tables.

        • The inconsistent keys are stored in the key table.

        • Details of inconsistent data of the hash, set, zset, and list types are stored in the field table.