PolarDB:CDC node test

Background information

CDC nodes in a PolarDB-X instance are used as log nodes to provide the binary logging service that is compatible with MySQL binary logs. You can use CDC nodes to subscribe to and consume binary logs. This can meet your business requirements in different scenarios.PolarDB-X This topic describes how to use TPC Benchmark C (TPC-C), Sysbench, and Large Transaction to test the performance of CDC nodes. Metrics, especially metrics on data latency and data throughput, measure the performance of CDC nodes. The following table describes the details of the metrics. For more information, see Log resource monitoring.

Test design

• Instance specifications for a CDC node test

  Node type	Node specifications	Number of nodes
  Compute node	32 vCPUs, 128 GB of memory	12
  Data node	32 vCPUs, 128 GB of memory	16
  CDC node	16 vCPUs, 32 GB of memory	2

• ECS instance type for stress testing

  ecs.hfc6.16xlarge (64 vCPUs, 128 GB of memory)

  Note To generate large volumes of traffic, you may need more than one Elastic Compute Service (ECS) instance.

Procedure

• TPC-C
  For information about the test procedure, see TPC-C test. In this example, the following settings are configured:

  • In the props.mysql file, set the warehouses parameter to 2000.
  • In the props.mysql file, set the runMins parameter to 10.
  • In the runBenchmark.sh file, set the Java virtual machine (JVM) memory to -Xms60g -Xmx60g.
• Sysbench
  • oltp_write_only

    For information about the test procedure, see Sysbench test. In this example, the oltp_write_only scenario is used. You can modify the number of concurrent threads to generate different volumes of traffic. Run the following command to perform a stress test:

    sysbench --config-file='sysb.conf' --db-ps-mode='disable' --skip-trx='off' --mysql-ignore-errors='all'  --tables='16' --table-size='10000000' --threads={Number of concurrent threads} oltp_write_only run

  • oltp_insert

    For information about the test procedure, see Sysbench test. In this example, the oltp_insert scenario is used. You can modify the number of concurrent threads to generate different volumes of traffic. Run the following command to perform a stress test:

    sysbench --config-file='sysb.conf' --db-ps-mode='disable' --skip-trx='on' --mysql-ignore-errors='all'  --tables='16' --table-size='10000000' --threads={Number of concurrent threads} oltp_insert run

  • Large Transaction

    Download the large_transaction.sql script to construct different sizes of transactions for a stress test. If you insert 200 thousand pieces of data, a transaction of 500 MB is constructed.

Test results

• oltp_write_only

  Metric (QPS)	DT (Unit: milliseconds)	BPS (Unit: MB/s)	EPS	TPS
  106081.68	45	34	274068	17761
  215321.43	60	68	532190	35672
  302061.64	100	97	788396	50560
  408594.41	287	128	1023873	68702
  518068.92	420	152	1201216	84654

  
• oltp_insert

  Metric (QPS)	DT (Unit: milliseconds)	BPS (Unit: MB/s)	EPS	TPS
  50984.67	46	25	309384	50984.67
  106907.38	95	51	645040	106907.38
  155172.85	145	75	935367	155172.85
  206096.27	267	99	1230641	206096.27
  252280.43	275	123	1519752	252280.43
  306653.64	388	149	1843716	306653.64
  352615.63	920	173	2136066	352615.63

  

If the amount of test data is less than 10 GB, data remains in the memory when operations are performed on binary logs, such as sorting and merging. In this case, the BPS is 500 MB/s. If the amount of test data is larger than 10 GB, data is exchanged between the memory and disks when operations are performed on binary logs. In this case, the BPS is 350 MB/s.

Transaction size	DT (Unit: milliseconds)	BPS (Unit: MB/s)	EPS
0.5 GB	2000	500	24
1 GB	4800	500	24
2 GB	7800	500	24
3 GB	9400	500	24
4 GB	12500	500	24
5 GB	17000	500	24
10 GB	55000	350	22
20 GB	115000	350	22