This topic describes how to use multi-master sharded tables.
Background information
Multi-master sharded tables in PolarDB for MySQL are a new type of cloud-native database. They support automatic horizontal splitting, hybrid shared storage, and a shared-nothing architecture. Multi-master sharded tables also automatically distribute data across multiple database clusters based on preset sharding rules. This allows each node to handle both read and write operations and achieve true distributed processing power.
Through horizontal splitting, PolarDB can effectively scale out databases and linearly increase the overall concurrent throughput. The architecture of multi-master sharded tables also combines the benefits of on-demand resource allocation from shared storage databases to ensure maximum resource utilization.
Prerequisites
The PolarDB for MySQL cluster must be a Multi-master Cluster (Limitless) Edition that runs kernel version 8.0. For more information, see Custom purchase and Purchase a subscription cluster.
A privileged account must be created. For more information, see Create and manage database accounts.
You must be connected to the database cluster. For more information, see Connect to a database cluster.
DDL statements
CREATE DATABASE
You can use the CREATE DATABASE statement to create a database and specify its default properties, such as the default character set and collation rules. The syntax is fully compatible with MySQL. You can also use DIST_DB={'Y' | 'N'} to specify whether the database supports multi-master sharded tables in database sharding mode. The default value is 'N'.
Multi-master sharded tables do not support converting a regular database to a database with the DIST_DB attribute.
Syntax
CREATE {DATABASE | SCHEMA} [IF NOT EXISTS] db_name
[create_option] ...
create_option: [DEFAULT] {
CHARACTER SET [=] charset_name
| COLLATE [=] collation_name
| ENCRYPTION [=] {'Y' | 'N'}
| DIST_DB [=] {'Y' | 'N'}
}Example
Create a database named test1. This lets you create multi-master sharded tables in database sharding mode within the test1 database. The default number of database shards is 8.
CREATE DATABASE test1 DIST_DB='Y';CREATE TABLE
Database sharding mode
You can use the CREATE TABLE statement to create a table. For a database marked with DIST_DB='Y', you can create only multi-master sharded tables in database sharding mode. You can use the new keyword DBDISTRIBUTION to specify the distribution method for the multi-master sharded table. The number of database shards is fixed at 8.
Syntax
CREATE [TEMPORARY] TABLE [IF NOT EXISTS] tbl_name
(create_definition,...)
[table_options]
[dbdistribution_options]
dbdistribution_options:
DBDISTRIBUTION BY
{ [LINEAR] HASH(expr)
| [LINEAR] KEY [ALGORITHM={1 | 2}] (column_list)
| RANGE{(expr) | COLUMNS(column_list)}
| LIST{(expr) | COLUMNS(column_list)} }
[(dbdistribution_definition [, dbdistribution_definition] ...)]
dbdistribution_definition:
DBDISTRIBUTION distribution_name
[VALUES
{LESS THAN {(expr | value_list) | MAXVALUE}
|
IN (value_list)}]
[[STORAGE] ENGINE [=] engine_name]
[COMMENT [=] 'string' ]
[DATA DIRECTORY [=] 'data_dir']
[INDEX DIRECTORY [=] 'index_dir']
[MAX_ROWS [=] max_number_of_rows]
[MIN_ROWS [=] min_number_of_rows]
[TABLESPACE [=] tablespace_name]Examples:
Single database and single table
CREATE database test;
CREATE TABLE t1(
id bigint NOT NULL,
name varchar(30),
PRIMARY key(id)
);Multi-master sharded table that supports database sharding mode
Use a hash function for splitting
Create a multi-master sharded table that is split into 8 database shards. The table is sharded by hashing the ID column, and the database shards are distributed across the nodes in the cluster.
CREATE TABLE dist1(
id bigint NOT NULL,
c1 int,
c2 varchar(30),
PRIMARY key(id)
) dbdistribution BY hash(id);LIST sharding
list is a set of enumerated values that can meet the needs of enumeration-based sharding. In list sharding, you can use DBDISTRIBUTION BY LIST(expr) to define the sharding rule. expr stands for expression and represents a column's value or an expression based on a column's value. expr must return an integer. You can then define each shard using VALUES IN (value_list), where value_list is a comma-separated list of integers.
CREATE TABLE t1 (
id INT NOT NULL,
store_id INT,
partion_no INT,
PRIMARY key(id, partion_no)
) DBDISTRIBUTION BY LIST(partion_no) (
DBDISTRIBUTION a VALUES IN (1),
DBDISTRIBUTION b VALUES IN (2)
);LIST-COLUMNS sharding
list column sharding is similar to list sharding. You can use DBDISTRIBUTION BY LIST COLUMNS(column_list) to define it, where column_list can be one or more column names, not an expression. The columns() function does not allow expressions other than column names. list columns sharding is not limited to integer columns. The applicable types include the following:
All integer types: TINYINT, SMALLINT, MEDIUMINT, INT(INTEGER), and BIGINT.
Other numeric types: For example, DECIMAL and FLOAT are not supported.
Date types: DATE and DATETIME.
String types: CHAR, VARCHAR, BINARY, and VARBINARY. TEXT and BLOB are not supported.
CREATE TABLE t3 (
id INT NOT NULL,
store_id INT,
partion_no varchar(20),
PRIMARY key(id, partion_no)
) DBDISTRIBUTION BY LIST COLUMNS (partion_no) (
DBDISTRIBUTION a values in ('id_1'),
DBDISTRIBUTION b values in ('id_2')
);ALTER TABLE
Compatible with MySQL syntax. For more information, see the official MySQL website.
CREATE INDEX
Compatible with MySQL syntax. For more information, see the official MySQL website.
Distributed query
For SQL queries that do not have a filter condition on the distributed key or that span multiple database or table shards, you can route them to a global read-only node. The node automatically aggregates data from all partitions. This lets you read all data without extra storage or synchronization links.
Global auto-increment column
A globally unique sequence is supported to generate values for a natural number sequence. However, these values are not guaranteed to be consecutive or monotonically increasing. It uses a batch allocation method where the primary node allocates a range of values each time. Therefore, when multiple primary nodes insert data at the same time, the generated auto-increment IDs may not be consecutive. Each time a batch is retrieved from the storage engine, the system first retrieves the last allocated ID and then retrieves a new batch of IDs to define a range.
CREATE TABLE t1( id int NOT NULL auto_increment, c1 int, c2 int, PRIMARY key(id, c1)) dbdistribution BY RANGE (c1) ( dbdistribution p0 VALUES less than (100), dbdistribution p1 VALUES less than (1000), dbdistribution p2 VALUES less than (10000), dbdistribution p3 VALUES less than (20000) );
Query OK, 0 rows affected (0.26 sec)
INSERT INTO t1 (c1, c2) VALUES (1, 1);
Query OK, 1 row affected (0.04 sec)MySQL single-node feature | Multi-master sharded table using a sequence |
One auto-increment column per table | Supported. |
An index must be on the auto-increment column | The auto-increment column cannot also be the distributed key. If it is, you must explicitly provide the auto-increment column value for each insert. However, a unique index is supported. |
Record insertion without specifying the column value | Supported. |
Record insertion with a specified column value | Supported only if the specified column value is within the local sequence range. Otherwise, it relies on the global index for duplication checks. |
Globally unique and monotonically increasing | Supports global uniqueness and maintains monotonic increase within a partition. |
Gaps in the sequence | The sequence has gaps. |
Modify the auto-increment counter | Supported. Note The sequence cache on each local node must be invalidated. |
Update auto-increment column values | Not supported. Note The current implementation has the same behavior as inserting a record with a specified column value. |
Development limitations
The following DDL syntax limitations apply only to multi-master sharded tables:
Operation | Usage constraint |
The name of a new table cannot contain | - |
| Not supported. |
| Not supported. |
| Not supported. |
| Not supported. |
| Not supported. |
| Not supported. |