Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Use YCSB and ApsaraDB for MongoDB to Monitor and Test Connection Parameters

Use YCSB and ApsaraDB for MongoDB to Monitor and Test Connection Parameters

This article describes how to use the stress testing tool YCSB to simulate the read and write traffic of ApsaraDB for MongoDB to observe the read/write splitting effect.

Background

This article presents a method for using the stress testing tool YCSB to simulate read and write traffic in ApsaraDB for MongoDB in order to observe the effectiveness of read/write splitting.

If the current application is using PyMongo or a similar method to access ApsaraDB for MongoDB, but the read/write splitting is not functioning as expected, this method can be used as a reference for the current database connection configuration or as a comparison to help identify any potential issues in the current environment. For example, this article can assist in troubleshooting the following problem:

The application connects to the ApsaraDB for MongoDB instance through PyMongo. In the current environment, the Connection String URI in the script for connecting to the instance has been configured with the readPreference=secondary parameter. However, it has been observed that the read traffic does not reach the secondary node of the ApsaraDB for MongoDB sharded cluster or replica set instance as expected.

Problem analysis:

Under normal circumstances, when the readPreference=secondary parameter is configured in the Connection String URI, read traffic should unconditionally be directed to the secondary node. If the actual situation does not meet expectations, it is necessary to consider the client/server perspective and troubleshoot from the following directions:

• The ApsaraDB for MongoDB node is not being connected correctly, and adjustments must be made to the database connection script.

• The client driver used to connect to the database is experiencing abnormalities (environmental issues).

• The method of viewing the workload of the instance node is incorrect, resulting in an incorrect perception of the situation.

Customers can use YCSB to simulate read traffic and observe if it reaches the secondary node.

By following this article and conducting tests using a different approach from the original environment, the root cause of the problem can be identified.

Required Resources

  1. The 4C8G replica set instance of ApsaraDB for MongoDB 4.4 and one read-only node.
  2. The ECS 8C32G instance and operating system CentOS (It is used to install YCSB and is in the same VPC as the ApsaraDB for MongoDB instance).

Test

1. Install YCSB

Log in to the ECS and install YCSB in the ECS.

curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.15.0/ycsb-0.15.0.tar.gztar xfvz ycsb-0.15.0.tar.gzcd ycsb-0.15.0

2. Modify the Configuration File

According to the actual test scenario, modify the configuration file, such as writing 0.1 billion data records and performing 10 million read operations.

vi workloads/workloada
recordcount=100000000 # The number of data recordsoperationcount=10000000 # The number of executionsworkload=com.yahoo.ycsb.workloads.CoreWorkloadreadallfields=truereadpromise=1 # Readupdateproportion=0 # Updatescanproportion=0insertproportion=0 # Insertrequestdistribution=zipfian

3. Prepare Test Data

cd /root/ycsb-0.15.0/binnohup ./ycsb load mongodb -s -P /root/ycsb-0.15.0/workloads/workloada -p mongodb.url=mongodb://root:xxxxxx@dds-3nsa957cbc4807343.mongodb.rds.aliyuncs.com:3717,dds-3nsa957cbc4807341.mongodb.rds.aliyuncs.com:3717,dds-3nsa957cbc4807342.mongodb.rds.aliyuncs.com:3717/admin?readPreference=secondary&w=0&replicaSet=mgset-71093327 &

4. Execute Read Operations

nohup ./ycsb run mongodb -s -P /root/ycsb-0.15.0/workloads/workloada -p mongodb.url=mongodb://root:xxxxxx@dds-3nsa957cbc4807343.mongodb.rds.aliyuncs.com:3717,dds-3nsa957cbc4807341.mongodb.rds.aliyuncs.com:3717,dds-3nsa957cbc4807342.mongodb.rds.aliyuncs.com:3717/admin?readPreference=secondary&w=0&replicaSet=mgset-71093327 &

5. View Running Logs to Ensure Normal Operation

tail -f nohup.out

6. View the ApsaraDB for MongoDB Monitoring

a. Insert Traffic on the Primary Node

1

b. Query Traffic on the Secondary Node

2

c. Query Traffic on the Read-Only Node

3

7. Use Mongostat to View Traffic

4
5
6

Summary

The read traffic is simulated using YCSB, and it successfully reaches the secondary node as expected. If the method of viewing the traffic is the same as described in this article, it indicates that the server (ApsaraDB for MongoDB instance) is functioning normally, and the issue lies with the client. For example, there may be an abnormality in the connection driver PyMongo.

MongoDB Database Stress testing monitoring ApsaraDB for MongoDB Tutorials YCSB PyMongo Stress Testing Tool
0 1 0
Share on

Read previous post:

About Database Kernel | PolarDB Query Optimization: Predicate Pushdown

Read next post:

Scenario Analysis | How PolarDB-X Supports SaaS Multi-Tenancy

ApsaraDB

459 posts | 99 followers

You may also like

Comments

ApsaraDB

459 posts | 99 followers

Related Products

More Posts by ApsaraDB

See All