This article presents a method for using the stress testing tool YCSB to simulate read and write traffic in ApsaraDB for MongoDB in order to observe the effectiveness of read/write splitting.
If the current application is using PyMongo or a similar method to access ApsaraDB for MongoDB, but the read/write splitting is not functioning as expected, this method can be used as a reference for the current database connection configuration or as a comparison to help identify any potential issues in the current environment. For example, this article can assist in troubleshooting the following problem:
The application connects to the ApsaraDB for MongoDB instance through PyMongo. In the current environment, the Connection String URI in the script for connecting to the instance has been configured with the
readPreference=secondary
parameter. However, it has been observed that the read traffic does not reach the secondary node of the ApsaraDB for MongoDB sharded cluster or replica set instance as expected.
Problem analysis:
Under normal circumstances, when the readPreference=secondary
parameter is configured in the Connection String URI, read traffic should unconditionally be directed to the secondary node. If the actual situation does not meet expectations, it is necessary to consider the client/server perspective and troubleshoot from the following directions:
• The ApsaraDB for MongoDB node is not being connected correctly, and adjustments must be made to the database connection script.
• The client driver used to connect to the database is experiencing abnormalities (environmental issues).
• The method of viewing the workload of the instance node is incorrect, resulting in an incorrect perception of the situation.
Customers can use YCSB to simulate read traffic and observe if it reaches the secondary node.
By following this article and conducting tests using a different approach from the original environment, the root cause of the problem can be identified.
Log in to the ECS and install YCSB in the ECS.
curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.15.0/ycsb-0.15.0.tar.gztar xfvz ycsb-0.15.0.tar.gzcd ycsb-0.15.0
According to the actual test scenario, modify the configuration file, such as writing 0.1 billion data records and performing 10 million read operations.
vi workloads/workloada
recordcount=100000000 # The number of data recordsoperationcount=10000000 # The number of executionsworkload=com.yahoo.ycsb.workloads.CoreWorkloadreadallfields=truereadpromise=1 # Readupdateproportion=0 # Updatescanproportion=0insertproportion=0 # Insertrequestdistribution=zipfian
cd /root/ycsb-0.15.0/binnohup ./ycsb load mongodb -s -P /root/ycsb-0.15.0/workloads/workloada -p mongodb.url=mongodb://root:xxxxxx@dds-3nsa957cbc4807343.mongodb.rds.aliyuncs.com:3717,dds-3nsa957cbc4807341.mongodb.rds.aliyuncs.com:3717,dds-3nsa957cbc4807342.mongodb.rds.aliyuncs.com:3717/admin?readPreference=secondary&w=0&replicaSet=mgset-71093327 &
nohup ./ycsb run mongodb -s -P /root/ycsb-0.15.0/workloads/workloada -p mongodb.url=mongodb://root:xxxxxx@dds-3nsa957cbc4807343.mongodb.rds.aliyuncs.com:3717,dds-3nsa957cbc4807341.mongodb.rds.aliyuncs.com:3717,dds-3nsa957cbc4807342.mongodb.rds.aliyuncs.com:3717/admin?readPreference=secondary&w=0&replicaSet=mgset-71093327 &
tail -f nohup.out
The read traffic is simulated using YCSB, and it successfully reaches the secondary node as expected. If the method of viewing the traffic is the same as described in this article, it indicates that the server (ApsaraDB for MongoDB instance) is functioning normally, and the issue lies with the client. For example, there may be an abnormality in the connection driver PyMongo.
About Database Kernel | PolarDB Query Optimization: Predicate Pushdown
Scenario Analysis | How PolarDB-X Supports SaaS Multi-Tenancy
ApsaraDB - March 26, 2024
ApsaraDB - September 8, 2021
- December 5, 2017
Alibaba Clouder - March 29, 2019
Alibaba Clouder - February 21, 2020
Alibaba Clouder - September 6, 2019
A secure, reliable, and elastically scalable cloud database service for automatic monitoring, backup, and recovery by time point
Learn MoreProvides comprehensive quality assurance for the release of your apps.
Learn MorePenetration Test is a service that simulates full-scale, in-depth attacks to test your system security.
Learn MoreMulti-source metrics are aggregated to monitor the status of your business and services in real time.
Learn MoreMore Posts by ApsaraDB