Overview of the data archiving feature - ApsaraDB for MongoDB

In the use of an online database, an increasing amount of data is stored in the database. The data that is infrequently accessed occupies much storage space and affects the query performance and business operation. In this case, you can use the data archiving feature provided by Data Management (DMS) to periodically archive data of specific tables in the database to other databases or storage space. You can also perform other operations to reduce the online storage cost. For example, you can delete the archived data from the source tables and optimize the storage usage of the source tables.

Background information

On e-commerce platforms, a large amount of order data is generated every day, and the need for access to these order data diminishes over time. To improve database performance and free up storage space, you can use the data archiving feature of DMS to periodically archive data.

Usage notes

The business that is running may be affected when you use the data archiving feature.
Important
To minimize the impact, we recommend that you use the data archiving feature during off-peak hours.
The data archiving task may fail if the tables to be archived contain large fields of the BLOB, TEXT, and other types and rows with large data sizes.
The data archiving feature is available only in the Singapore and Indonesia (Jakarta) regions.
You cannot archive data from tables that contain virtual columns.
You cannot archive data from a database hosted on a third-party cloud service or a self-managed database in the DMS console or by using API operations.

Comparison between data archiving of DMS and data migration of DTS

Features
- Data archiving of DMS allows you to configure filter conditions to archive infrequently accessed data to other storage space. This reduces the load on the primary database and improves database performance.
- Data migration of Data Transmission Service (DTS) allows you to migrate data between homogeneous and heterogeneous data sources. For more information, see Overview of data migration scenarios.
Scenarios
- Data archiving of DMS allows you to archive infrequently accessed data to other databases or storage space on a regular basis or at a time.
- Data migration of DTS is applicable to scenarios such as data migration to Alibaba Cloud, data migration between databases within Alibaba Cloud, and database splitting or scale-out.

Comparison among different categories of archiving destinations

Databases

Item	Lindorm	AnalyticDB for MySQL V3.0	AnalyticDB for PostgreSQL	ApsaraDB RDS for MySQL	PolarDB for MySQL
Supported source database types	MySQL: ApsaraDB RDS for MySQL, PolarDB for MySQL, and AnalyticDB for MySQL V3.0 PostgreSQL: ApsaraDB RDS for PostgreSQL and PolarDB for PostgreSQL PolarDB-X 2.0 Note The account of a MySQL database must have the REPLICATION CLIENT permission.
Billing	You can use the data archiving feature free of charge. However, you may be charged when you purchase and use the destination instance.
Deletion of archived data from sources	After you archive data to a destination database, DMS can automatically delete the archived data from the source tables based on your settings. You no longer need to delete the source data after the data is archived. This reduces data risks. The source table data is deleted without the lock of source tables. The process is stable and fast and does not involve large transactions. The source data is cached in a local temporary table. You can restore the cached data if necessary.
Query of archived data	You can query the archived data in the DMS console. You can also query the archived data in the corresponding database engine.
Configuration of the data archiving destination	You must specify a destination instance. You do not need to specify a destination database or destination table. DMS automatically creates a destination database and a destination table based on the names of the source database and source table.
Change of the table schema	A column is added to the archived tables in the destination to store information such as the ticket ID and archiving time. Data in the tables can be used as usual.
Usage	Merge multiple tables into a large wide table to improve query performance. Support highly concurrent requests to query data online in the scenarios where a Lindorm instance has 2 CPU cores. For example, you can query the historical data of bills and orders. Support the adaptive compression and automatic hot and cold data separation features for full data and reduce storage costs.	Update data within milliseconds and query data within sub-seconds. Support high-throughput offline processing and high-performance online analysis to reduce costs and improve efficiency.	Analyze data. Reduce online storage costs.	Automatically synchronize data to analytic instances to accelerate complex analysis and queries. Reduce storage costs.	Analyze data. Reduce online storage costs.
Data control	The strongest data control capability is provided. DMS can archive data to a database instance that you manage. You can flexibly process data by interacting with the databases.

Storage space

Item	Dedicated storage space (recommended)	Custom OSS bucket	OSS bucket connected to DBS
Supported source database types	MySQL: ApsaraDB RDS for MySQL, PolarDB for MySQL, and AnalyticDB for MySQL V3.0 PostgreSQL: ApsaraDB RDS for PostgreSQL and PolarDB for PostgreSQL MongoDB 3.6 and later Note The account of a MySQL database must have the REPLICATION CLIENT permission.	MySQL: ApsaraDB RDS for MySQL, PolarDB for MySQL, and AnalyticDB for MySQL V3.0 PostgreSQL: ApsaraDB RDS for PostgreSQL and PolarDB for PostgreSQL MongoDB 3.6 and later PolarDB-X 2.0	MySQL: ApsaraDB RDS for MySQL and PolarDB for MySQL PolarDB for PostgreSQL PolarDB-X 2.0
Billing	You are not charged for archiving data to dedicated storage space during public preview.	You can use the data archiving feature free of charge. However, you may be charged when you purchase and use Object Storage Service (OSS) buckets.	DMS creates a backup schedule in Database Backup (DBS). You are charged for data backup and storage based on the amount of data that is backed up. For more information about the billing, see Billing overview. Note By default, an xlarge backup schedule is created in DBS.
Deletion of archived data from sources	After you archive data to a destination database, DMS can automatically delete the archived data from the source tables based on your settings. You no longer need to delete the source data after the data is archived. This reduces data risks. The source table data is deleted without the lock of source tables. The process is stable and fast and does not involve large transactions. The source data is cached in a local temporary table. You can restore the cached data if necessary.		The archived data cannot be automatically deleted from the source database. You must create a regular data change ticket to manually delete the archived data from the source database.
Query of archived data	You can query the archived data by using the logical data warehouses in DMS. You can also query the archived data by using the file merging feature to achieve higher efficiency.	You can query the archived data by using the logical data warehouses in DMS.	-
Configuration of the data archiving destination	You do not need to focus on the archiving destination.	You must specify an OSS bucket.	You do not need to specify an OSS bucket. DMS automatically creates a backup schedule in DBS to archive data to OSS.
Change of the table schema	The table schema is not changed.		The table schema is not changed.
Usage	Analyze data. Reduce some online storage costs.	Reduce the most online storage costs.	Reduce many online storage costs.
Data control	Strong data control capability is provided. Data is archived and stored in DMS. You do not need to focus on the archiving destination.	Strong data control capability is provided. DMS can archive data to an OSS bucket that you purchase.	Moderate data control capability is provided. DMS can archive data to an OSS bucket connected to DBS.

References

FAQ

Q: If I select Clean up the archived data of the original table (delete-No Lock) when I configure parameters for data archiving, is the historical data still available for retrieval?

A: Yes, the historical data is still available for retrieval even if you select Clean up the archived data of the original table (delete-No Lock) when you configure parameters for data archiving. The data archiving feature regularly archives the data of large tables to other databases. You can query archived historical data in the destination instances to which the data is archived.

ApsaraDB for MongoDB:Data archiving