DataWorks:Change data source from MySQL to PolarDB via API

Limitations

This method only supports changing the source from a MySQL data source to a PolarDB data source. Other data source type conversions are not supported.

Prerequisites

Before you begin, make sure that you have:

• Java Development Kit (JDK) installed

• A Maven-based Java project

• An Alibaba Cloud account with an AccessKey ID and AccessKey secret

• A DataWorks workspace with the target data synchronization task already created

• The PolarDB data source already registered in your DataWorks workspace

• Sufficient permissions to call DataWorks API operations (ListFiles, GetFile, UpdateDISyncTask, SubmitFile, DeployFile, ListNodes, RunCycleDagNodes)

API call sequence

The following diagram shows the API call sequence:

ListFiles          Find the sync task file by path and name
    |
GetFile            Retrieve the current JSON configuration
    |
modifyContent      Replace MySQL reader config with PolarDB (local logic)
    |
UpdateDISyncTask   Save the updated configuration to DataWorks
    |
SubmitFile         Submit the file for deployment review
    |
GetDeployment      Check submission status
    |
DeployFile         Deploy the submitted file to production
    |
GetDeployment      Confirm deployment status
    |
ListNodes          Look up the node ID by node name
    |
RunCycleDagNodes   Trigger a retroactive run of the updated task

Each step is required because DataWorks separates the development lifecycle into distinct phases: edit, submit, deploy, and run. Skipping any step causes the change to remain in draft state or not take effect in production.

Add Maven dependencies

Add the following dependencies to your pom.xml file:

<dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>aliyun-java-sdk-core</artifactId>
    <version>4.5.20</version>
</dependency>
<dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>aliyun-java-sdk-dataworks-public</artifactId>
    <version>3.4.4</version>
</dependency>

Configuration changes

The modifyContent method finds the reader step (where category is reader and stepType is mysql) and updates three fields:

All other fields remain unchanged, including columns, table names, writer settings, and speed settings.

{
    "stepType": "mysql",
    "parameter": {
        "envType": 0,
        "datasource": "mysql_from_polardb",
        "column": [
            "id",
            "name",
            "create_time",
            "create_user"
        ],
        "tableComment": "Test",
        "connection": [
            {
                "selectedDatabase": "polardb_db1",
                "datasource": "mysql_from_polardb",
                "table": [
                    "lcl_test_demo"
                ]
            }
        ],
        "where": "",
        "splitPk": "id",
        "encoding": "UTF-8"
    },
    "name": "Reader",
    "category": "reader"
}

{
    "stepType": "polardb",
    "parameter": {
        "envType": 0,
        "datasource": "polardb",
        "column": [
            "id",
            "name",
            "create_time",
            "create_user"
        ],
        "tableComment": "Test",
        "connection": [
            {
                "selectedDatabase": "polardb_db1",
                "datasource": "polardb",
                "table": [
                    "lcl_test_demo"
                ]
            }
        ],
        "where": "",
        "splitPk": "id",
        "encoding": "UTF-8"
    },
    "name": "Reader",
    "category": "reader"
}

Complete Java sample code

The following code demonstrates the full workflow: locate the task, retrieve and modify its configuration, deploy the changes, and trigger a run.

Replace the following placeholder values before running:

API operations reference

Verify the result

After running the code:

1. Check deployment status. The getDeployment output should show a success status. If the status indicates failure, check whether the PolarDB data source is correctly registered in your DataWorks workspace.

2. Verify in the DataWorks console. Open the data synchronization task in Data Development and confirm that the reader source now shows PolarDB instead of MySQL.

3. Check the retroactive run. In Operation Center, find the task instance triggered by RunCycleDagNodes and verify that it completed successfully with data written to the MaxCompute destination table.

Troubleshooting

Usage notes

• The projectId (1911L in the sample) must match your DataWorks workspace ID. Find this value in the DataWorks console on the Workspace Details page.

• The Thread.sleep(10000) calls (10 seconds) are simple wait intervals. In production, poll GetDeployment in a loop until the status indicates completion or failure, rather than relying on fixed wait times.

• Store your AccessKey ID and AccessKey secret securely. Do not hardcode credentials in source files. Use environment variables or a credentials provider instead.

• The endpoint follows the pattern dataworks.{regionId}.aliyuncs.com. Replace {regionId} with the region of your workspace.