Confidential Database Performance Test Report - PolarDB - Alibaba Cloud Documentation Center

This topic describes the performance of a Confidential Database under different encryption conditions to help you evaluate and select an encryption solution.

Test environment

Configuration item

ECS instance (test client deployed on the ECS instance)

PolarDB for MySQL cluster

Description

Region and zone

Hangzhou Zone H (primary), Hangzhou Zone J

Hangzhou Zone H

The test was performed in the same region and zone.

Network type

Virtual Private Cloud (VPC)

The test was performed in the same VPC.

CPU and memory

8 cores, 32 GB

64 cores, 512 GB

Not applicable

Instance type

ecs.s6-c1m4.2xlarge

polar.mysql.x8.8xlarge

Not applicable

Instance/Image version

Alibaba Cloud Linux 2.1903 LTS 64-bit Quick Launch

Kernel version: 8.0.2.2.12

Not applicable

Test tool (oltpbench)

Oltpbench is an open source benchmark tool for Online Transactional Processing (OLTP) that tests the performance and scalability of database management systems. It provides multiple benchmark programs, such as TPC-C, TPC-E, and YCSB, to simulate different workload types. This topic uses the TPC-C test from oltpbench to evaluate the performance of a Confidential Database in OLTP scenarios.

Metric

Transactions Per Second (TPS): The number of transactions that a database executes per second. This is measured by the number of successful COMMIT operations.

Test steps

A confidential database encrypts data based on the encryption rules that you set. This test performs a what-if analysis of the confidential database's performance in different scenarios by adjusting the encrypted data columns in the TPC-C test.

Configure the oltpbench project on your ECS instance. For more information, see Configure oltpbench.
Integrate EncJDBC with oltpbench. For more information, see Integrate EncJDBC.

Create an oltpbench configuration file as follows:

<?xml version="1.0"?>
<parameters>

    <!-- Connection details -->
    <dbtype>mysql</dbtype>
    <driver>com.aliyun.encdb.mysql.jdbc.EncDriver</driver>
    <DBUrl>jdbc:mysql:encdb://xxxx.rwlb.rds.aliyuncs.com:3306/xxxx</DBUrl>
    <DBName>xxxx</DBName>
    <username>xxxx</username>
    <password>xxxxx</password>
    <isolation>TRANSACTION_SERIALIZABLE</isolation>

    <scalefactor>1</scalefactor>

    <!-- The workload -->
    <terminals>8</terminals>
    <works>
        <work>
            <time>180</time>
            <rate>unlimited</rate>
            <weights>45,43,4,4,4</weights>
        </work>
    </works>


    <!-- TPCC specific -->
        <transactiontypes>
        <transactiontype>
                <name>NewOrder</name>
        </transactiontype>
        <transactiontype>
                <name>Payment</name>
        </transactiontype>
        <transactiontype>
                <name>OrderStatus</name>
        </transactiontype>
        <transactiontype>
                <name>Delivery</name>
        </transactiontype>
        <transactiontype>
                <name>StockLevel</name>
        </transactiontype>
        </transactiontypes>
</parameters>

After compiling the oltpbench project, run the following command to import data into the database.
```
./oltpbenchmark -b tpcc -c tpcc_encmysql.xml --load=true --create=true -s 1 -o tpcc
```
Repeatedly adjust the encryption settings of the Confidential Database, TPC-C parameters, and other configurations to comprehensively test the performance (TPS/QPS) in OLTP scenarios.
```
./oltpbenchmark -b tpcc -c tpcc_encmysql.xml --execute=true -s 1 -o tpcc
```

Test scenarios and results

Scenario description

Each test result is the average TPS from three test runs. Each test run lasts for one minute.
The performance test results include the overhead of decrypting returned encrypted fields on the client.
In this TPC-C test scenario, Queries Per Second (QPS) is a fixed multiple of TPS, at approximately 9 to 10 times the TPS value. Therefore, this test uses only TPS to measure performance.
The Terminal parameter simulates the number of concurrent clients in TPC-C, which corresponds to the number of parallel test threads.

Scenario 1: Encrypt 20% of business data columns

In this scenario, columns that contain ID information were prioritized for encryption. This simulates the requirement to encrypt data such as ID card numbers and order numbers in real-world business scenarios.

Test results

Number of terminals	Plaintext (TPS)	20% sensitive columns encrypted (TPS)	Performance overhead
8	260.55	237.64	8.79%
16	302.78	280.28	7.43%
24	330.04	298.03	9.70%
32	343.65	315.25	8.26%

The test results show that when 20% of sensitive business data columns are encrypted, the performance overhead of the Confidential Database in the TPC-C test is between 7% and 9% compared to a plaintext database.

Scenario 2: Encrypt 50% of business data columns

In addition to the ID information columns encrypted in Scenario 1, other sensitive business information columns were also encrypted. These columns include data such as price, date, and quantity.

Test results

Number of terminals	Plaintext (TPS)	50% sensitive columns encrypted (TPS)	Performance overhead
8	260.55	221.01	15.18%
16	302.78	256.64	15.24%
24	330.04	273.53	17.12%
32	343.65	293.65	14.55%

The test results show that when 50% of sensitive business data columns are encrypted, the performance overhead of the Confidential Database in the TPC-C test is between 14% and 17% compared to a plaintext database.

Scenario 3: Encrypt all business data columns

Test results

Number of terminals	Plaintext (TPS)	100% sensitive columns encrypted (TPS)	Performance overhead
8	260.55	201.62	22.62%
16	302.78	238.45	21.25%
24	330.04	256.36	22.32%
32	343.65	267.92	22.04%

The test results show that when all business data columns are encrypted, the performance overhead of the Confidential Database in the TPC-C test is between 21% and 22% compared to a plaintext database.