Verify migrated Redis data

Prerequisites

• Redis data is migrated.

• An Elastic Compute Service (ECS) Linux instance is created for running the redis-full-check tool. For more information, see Create an ECS instance.

• The ECS instance can access the source and destination Redis instances.

  Note

  • If the ECS instance and the source or destination Redis instance are in the same virtual private cloud (VPC), you must add the internal IP address of the ECS instance to a whitelist of the Redis instance. For more information, see Configure whitelists.

  • If the ECS instance and the source or destination Redis instance are not in the same VPC, the ECS instance can access the Redis instance over the public endpoint. For more information, see Use a public endpoint to connect to an instance.

Scenario

If an exception occurs during Redis data migration, the data is inconsistent between the source and destination Redis instances. You can use the redis-full-check tool to locate abnormal data, which provides a reliable basis for data alignment.

redis-full-check is a Redis data verification tool developed in-house by Alibaba Cloud. It can extract data from the source and destination instances, conduct multiple rounds of comparisons of the extracted data, and then record the comparison results in an SQLite3 database. This tool can be used to verify full data.

Procedure

1. Log on to the ECS instance. For more information, see Connect to an instance.

2. Download the redis-full-check tool on the ECS instance.

   Note

   We recommend that you download the latest version of redis-full-check.

3. Run the following command to decompress the redis-full-check.tar.gz package:

   tar -xvf redis-full-check.tar.gz

4. Run the following command to verify data:

   ./redis-full-check -s "<Endpoint 1 of the source Redis instance:Port number of endpoint 1;Endpoint 2 of the source Redis instance:Port number of endpoint 2;Endpoint 3 of the source Redis instance:Port number of endpoint 3;>" -p <Password of the source Redis instance> -t <Endpoint of the destination Redis instance:Port number> -a <Password of the destination Redis instance> --m=1 --sourcedbfilterlist=0;2 --targetdbfilterlist=0;2

   Sample command:

   ./redis-full-check -s "10.xx.xx.1:6379" -p SourcePwd233 -t "r-bp1zc********.redis.rds.aliyuncs.com:6379" -a testaccount:Test1234 --m=1 --sourcedbfilterlist=0;2 --targetdbfilterlist=0;2

   The following table describes common parameters of the redis-full-check tool. For more information, see Configure redis-full-check.

   Table 1. Common parameters of redis-full-check

   Parameter	Description	Example
   -s	The endpoint and port of the source Redis instance. Note If the source Redis instance is a cluster, separate cluster endpoints with semicolons (;). Enclose the cluster endpoints in a pair of double quotation marks ("). This parameter is required.	r-bp1xxxxxxxxxxxxx.redis.rds.aliyuncs.com:6379 "10.xx.xx.1:7000;10.xx.xx.1:7001;10.xx.xx.2:7002;10.xx.xx.2:7003"
   -p	The username and password that are used to connect to the source Redis instance. Specify this parameter in the <username>:<password> format. Example: testaccount:Test1234. If no username is set for the instance, you can enter only the password. If no password is set for the instance, you can leave the parameter empty.	SourcePwd233 or testaccount:Test1234
   -t	The endpoint and port of the destination Redis instance. Note If the destination Redis instance is a cluster, separate cluster endpoints with semicolons (;). Enclose the cluster endpoints in a pair of double quotation marks ("). This parameter is required.	r-bp1xxxxxxxxxxxxx.redis.rds.aliyuncs.com:6379 "10.xx.xx.1:7000;10.xx.xx.1:7001;10.xx.xx.2:7002;10.xx.xx.2:7003"
   -a	The username and password that are used to connect to the destination Redis instance. Specify this parameter in the <username>:<password> format. Example: testaccount:Test1234. If no username is set for the instance, you can enter only the password. If no password is set for the instance, you can leave the parameter empty.	TargetPwd233 or testaccount:Test1234
   --sourcedbtype	The type of the source Redis instance. Default value: 0. Valid values: 0: standalone or standard 1: cluster (direct connection mode) 2: cluster (proxy mode)	--sourcedbtype=1
   --sourcedbfilterlist	The databases for which you want to verify data in the source Redis instance. Note This parameter is not required for open source Redis clusters. If you do not set this parameter, the data in all databases is verified. Separate multiple databases with semicolons (;).	--sourcedbfilterlist=0;1;2
   --targetdbtype	The type of the destination Redis instance. Valid values: 0: standalone or standard 1: cluster (direct connection mode) 2: cluster (proxy mode)	--targetdbtype=0
   --targetdbfilterlist	The databases for which you want to verify data in the destination Redis instance. Note This parameter is not required for open source Redis clusters. If you do not set this parameter, the data in all databases is verified. Separate multiple databases with semicolons (;).	--targetdbfilterlist=0;1;2
   -d	The name of the file for storing inconsistent keys. Default value: result.db.	xxx.db
   --comparetimes	The number of verifications. Default value: 3. Minimum value: 1. We recommend that you set this parameter to a value that does not exceed 5.	--comparetimes=1
   -m	The verification mode. Default value: 2. Valid values: 1: verifies full data. 2: only verifies the length of the value. 3: only checks whether the keys exist. 4: verifies full data except for large keys.	-m=1
   --qps	The queries per second (QPS). Default value: 15000. Note Minimum value: 1. Maximum value: depends on the server performance.	--qps=10
   --filterlist	The keys to verify. Separate multiple keys with vertical bars (|). Note abc*: matches all keys that start with abc. abc: matches the abc key.	--filterlist=abc*|efg|m*

   Note

   After the verification is completed, the comparison result is displayed on the CLI. The following result indicates that two keys are inconsistent between the source and destination Redis instances. If the number of inconsistent keys is 0, the data is consistent between the source and destination Redis instances.

   all finish successfully, totally 2 keys or fields conflict

5. Check the SQLite3 database that stores the inconsistent keys.

   1. Run the sqlite3 result.db.3 command.

      Note

      Inconsistent keys are stored in result.db.3 by default.

   2. Execute the SELECT * FROM key; statement.

      Figure 1. View the list of inconsistent keys

      Note

      The SQLite3 database contains the key and field tables.

      • The inconsistent keys are stored in the key table.

      • Details of inconsistent data of the hash, set, zset, and list types are stored in the field table.