This topic describes how to use DataWorks data integration to synchronize data from a database to Hologres in real time.
Prerequisites
Activate DataWorks. For more information, see Overview.
You have successfully activated ApsaraDB.
To learn if you can synchronize data across regions, see Network connectivity solutions.
Background information
Hologres is a real-time interactive analytics product. It seamlessly connects with the big data ecosystem and deeply integrates with DataWorks, an intelligent R&D platform. This integration supports data query and analysis with high concurrency and low latency. You can use DataWorks data integration to synchronize data from a database to Hologres in real time. Then, you can query, analyze, and process the data with high concurrency and low latency.
Common databases that support real-time data synchronization include Oracle, PolarDB, and PolarDB for MySQL.
For more information about supported data sources, see Supported data sources and synchronization solutions.
Related principles: MySQL Reader, Oracle data source, PolarDB Reader, SQL Server Reader, and Hologres data source.
Procedure
DataWorks data integration lets you synchronize data from various databases to Hologres in real time. This process is stable and efficient. Follow these steps to complete the task.
Configure the source data source
Before you synchronize data, you must configure the source data source. For example, to synchronize data from MySQL to Hologres in real time, you must configure a MySQL data source. Select and configure the data source that meets your requirements. For more information, see Configure a data source.
Configure the Hologres destination data source
NoteThe Hologres data source must use an exclusive resource group for data integration.
You must also configure the Hologres destination data source before you synchronize data. For more information, see Configure a Hologres data source.
Configure a task
After you configure the source and Hologres destination data sources, configure the synchronization method and run the task. DataWorks data integration provides three real-time synchronization methods. You can choose the method that best suits your business needs. The following table provides details.
Synchronization type
Scenario
Supported data sources
Data source configuration guide
Sync task configuration guide
Single-table real-time synchronization
Synchronizes data changes from some source tables to the destination database in real time. This keeps the destination database consistent with the source.
MySQL binary logging
DataHub
LogHub
Kafka
PolarDB
SQL Server
Configure real-time synchronization of incremental data from a single table
Full database real-time synchronization
Synchronizes data changes from all source tables to the destination database in real time. This keeps the destination database consistent with the source.
PolarDB for MySQL
PolarDB
MySQL
Synchronization solution
Provides solutions for various data synchronization scenarios between different data sources. These scenarios include real-time synchronization, offline full synchronization, and offline incremental synchronization. The solutions help you move data to the cloud with one click, efficiently and conveniently. The main features include the following:
Initial full synchronization of data.
Real-time writing of incremental data.
Scheduled automatic merge of incremental and full data into new full-table partitions.
PolarDB for MySQL
Oracle
MySQL
PolarDB-X
PostgreSQL
NoteWhen you use DataWorks to synchronize data to Hologres, you can also add extra fields to the Hologres destination table, such as an update_time field. For more information, see Configure and manage a real-time sync task.