what is the performance testing tool - shell - Lindorm - Alibaba Cloud Documentation Center

AHBench is a benchmark testing suite developed by the Alibaba Cloud Lindorm team. It supports one-click performance testing of Lindorm and HBase clusters.

Introduction

This testing suite integrates the Yahoo! Cloud Serving Benchmark (YCSB) tool. YCSB provides features such as test sets, test flow control, and result aggregation. You can perform one-click performance testing with a simple configuration.

Preparations

If the stress testing client is deployed on an ECS instance, ensure that the Lindorm instance and the ECS instance meet the following conditions for network connectivity. For more information about how to view instance information, see View instance information.
- The instances are in the same region. We recommend that they are in the same zone to reduce network latency.
- The instances have the same network type.
  Note
  - Virtual Private Cloud (VPC) offers higher security. We recommend that you use a VPC.
  - If the network type is VPC, the instances must use the same VPC ID.
Add the IP address of the stress testing client to the whitelist of the Lindorm instance. For more information, see whitelist.
Download the AHBench test suite, upload it to the stress testing client, and then extract it.

Usage notes

Stress testing may overload the system under test. Do not run this testing tool in a production environment.
This testing suite uses "ahbenchtest-read" and "ahbenchtest-write" as the names for test tables. These tables may be deleted and recreated during testing. Ensure that these tables can be safely deleted.
Ensure that the test cluster has sufficient storage space.
ECS provides a virtual runtime environment. Therefore, performance testing results for the same instance type may fluctuate by 5% to 10%. This fluctuation is within the expected range.

Runtime environment

Ensure that the runtime environment of the stress testing client meets the following requirements:

Linux
JDK 1.8 or later
Python 2.7
We recommend that the client has at least 16 dedicated CPU cores.

Configure cluster endpoint

Configure the endpoint of the Lindorm or HBase cluster to be tested in the AHBench/conf/hbase-site.xml file.

For more information about how to access Lindorm, see Develop applications using the HBase Java API. You also need to add the following parameters to the hbase-site.xml file.

<property>
  <name>hbase.client.connection.impl</name>
  <value>org.apache.hadoop.hbase.client.AliHBaseUEClusterConnection</value>
</property>

Configure environment variables

Configure the runtime environment variables for the tool in the AHBench/conf/ahbench-env.properties file.

Run the following command to open the ahbench-env.properties file.
```
vi AHBench/conf/ahbench-env.properties
```
Configure the JDK installation path. For example, JAVA_HOME=/usr/java/jdk1.8.0/. If Java is already in the system PATH, you can skip this configuration.
Configure the version of the HBase cluster under test. Enter 1 for version 1.x and 2 for version 2.x.
```
HBASE_VERSION=2
```

Configure test-related parameters (optional)

You can configure test-related parameters, such as compression, encoding, number of threads, data volume, and field size, in the AHBench/conf/ahbench-settings.properties file. By default, no modifications are needed. You can modify the parameters as required for specific scenarios.

Note

Some parameters are supported only by specific Lindorm versions. For example, ZSTD compression and INDEX encoding are supported only by Lindorm. You can configure ZSTD and INDEX to achieve better performance.

# Configure the compression algorithm for the tables under test. Options include the following:
# NONE LZO ZSTD SNAPPY GZ LZ4 ZSTD, etc.
# Note that some systems under test may not support the specified compression algorithm.
# Lindorm recommends using ZSTD.
ahbench.table.compression=SNAPPY

# Configure the encoding algorithm for the tables under test. Options include the following:
# NONE DIFF INDEX
# Note that some systems under test may not support the specified encoding algorithm.
# Lindorm recommends using INDEX.
ahbench.table.encoding=DIFF

Start testing

Quick test set

The test data volume is 10 million rows. The total runtime is approximately 40 minutes, but may vary depending on the system under test. This test requires at least 20 GB of storage space.
```
cd AHBench
./fast_test                 
```
Full test set

The test data volume is 2 billion rows. The total runtime is approximately 25 hours, but may vary depending on the system under test. This test requires at least 2 TB of storage space.
```
cd AHBench
./full_test
                    
```
To repeat this test, you can skip the data import phase to reduce the runtime, provided that the previous test ran successfully. Skipping the import phase reduces the total test runtime to approximately 3.5 hours. The actual runtime may vary depending on the system under test.
```
cd AHBench
./full_test --skipload
                    
```

Analyze test results

After the test is complete, a Comma-Separated Values (CSV) file is generated in the current directory. You can copy the test results to data analysis software, such as Excel or Numbers, for further comparison and analysis.

View the name of the CSV file.
```
ls -ltr
```
View the content of the specified CSV file.
```
cat full_throughput.csv
```

The following figure shows the content of a CSV file.

csvwenjian

FAQ

If the test exits with an error, verify the following:

JAVA_HOME is correctly set and the Python runtime environment is installed.
The endpoint of the cluster under test is correctly entered.
The HBase version of the cluster under test is correctly entered.
Indicates whether the cluster supports the specified compression algorithm.
The cluster under test is running normally.