Data Integration is a reliable, secure, cost-effective, elastic, and scalable data synchronization platform provided by Alibaba Cloud. It supports data storage across heterogeneous systems and offers offline (both full and incremental) data access channels in diverse network environments for more than 20 types of data sources. For more information, visit Supported data source types, Reader plug-ins, and Writer plug-ins.

Scenarios

  • Export data from AnalyticDB for PostgreSQL to other data sources and process the data as needed.
  • Import processed data from other data sources to AnalyticDB for PostgreSQL.

In both the preceding scenarios, you can execute a synchronization task in Data Integration. For more information about how to create and configure a synchronization task including the data source and whitelist, visit the DataWorks documentation. The following sections detail how to import data to and export data from AnalyticDB for PostgreSQL.

Preparations

Complete the following preparations for Data Integration:

  1. Register an Alibaba Cloud account.
  2. Activate MaxCompute. After a default MaxCompute data source is generated, log on to the DataWorks console by using your Alibaba Cloud account.
  3. Create a workspace. In the workspace, you can complete your workflow and maintain both your data and tasks.
Note If you want to create a synchronization task in Data Integration by using the credentials of your RAM user, you must grant the required permissions to your RAM user. For more information, visit Create a RAM user.

Complete the following preparations for AnalyticDB for PostgreSQL:

  1. If you want to import data, use the psql CLI client to create both the destination database and table on your AnalyticDB for PostgreSQL instance.
  2. If you want to export data, configure an IP address whitelist for your AnalyticDB for PostgreSQL instance. For more information, visit Add whitelist.

Import data

Add a data source in the DataWorks console. For more information, visit Configure data sources.

Configure a synchronization task.

The synchronization task is used to synchronize data from a data source to AnalyticDB for PostgreSQL. You can configure the synchronization task in wizard or script mode.

  • To configure a synchronization task in wizard mode, follow these steps:
    1. Create a data synchronization node.
    2. Specify a data source.
    3. Specify a data destination. This must your AnalyticDB for PostgreSQL instance.
    4. Configure the mappings between fields from the source and destination tables.
    5. Specify a maximum transmission rate and dirty data check rules.
    6. Configure scheduling attributes.
    Note For more information, visit Configure a synchronization task in wizard mode.
  • To configure a synchronization task in script mode, follow these steps:
    1. Create a data synchronization node.
    2. Import a template.
    3. Configure a reader for the synchronization task.
    4. Configure a writer for the synchronization task. The writer must be your AnalyticDB for PostgreSQL instance.
    5. Configure the mappings between fields from the source and destination tables.
    6. Specify a maximum transmission rate and dirty data check rules.
    7. Configure scheduling attributes.
    Note For more information, visit Configure a synchronization task in script mode.

Export data

The data export procedure is similar to the data import procedure, except that you must specify AnalyticDB for PostgreSQL as the data source. For more information, visit Add an AnalyticDB for PostgreSQL connection.

References

For more information, visit the DataWorks documentation.