Migration Assistant allows you to export different types of data objects in your workspace. The types of data objects include Periodic tasks, Manual tasks, Resources, Function, Table, Data source, Components, Temporary query, DATA_SERVICE, and Data Quality. This topic describes how to create and view export tasks.
Background information
Migration Assistant allows you to export data objects in full export, incremental export, or custom export mode. You can select an export mode based on your business requirements.
Full export tasks are used to export all or selected types of data objects in your workspace. For example, you can use a full export task to back up the code of all nodes in your workspace or clone your workspace for a test environment. When you run a full export task, data objects of the latest version are exported.
Saved, committed, or deployed data objects can be exported. If a node is saved in both the development environment and production environment, the node saved in the development environment is exported. For more information, see Create a full export task.
Incremental export tasks are used to export all or selected types of data objects that are modified after a specific date. For more information, see Create an incremental export task.
NoteYou cannot configure a blacklist for incremental export tasks.
Custom export tasks are used to export data objects that you specify in a fine-grained manner. For example, you can use a custom export task to extract a common workflow. This way, you can clone this workflow to other workspaces. If the production environment and development environment of your workspace are isolated, you can use a custom export task to export nodes in the development environment. This way, you can import the nodes to the production environment for deployment. For more information, see Create a custom export task.
Limits
The following table lists the limits on the Migration Assistant service of DataWorks in different editions.
Item
Basic Edition
Standard Edition
Professional Edition
Enterprise Edition
Maximum number of packages that can be exported within a tenant
NoteIf the number of packages that you want to export exceeds the maximum number of packages that can be exported in an edition, you must upgrade the DataWorks edition to meet your business requirements. You cannot pay to export the desired number of packages.
Cumulative quantity: 10
Cumulative quantity: 30
Cumulative quantity: 100
No upper limit
Maximum size of a package that you want to import
100M
300M
No upper limit
No upper limit
Support for automatic commit and deployment during data import
No
No
Yes
Yes
Only an Alibaba Cloud account or a RAM user to which the Workspace Administrator role is assigned can be used to create export or import tasks. If you use a RAM user to which the Workspace Administrator role is not assigned, you can only view export and import tasks.
When you export or import monitoring rules for Data Quality, take note of the following points:
When you export monitoring rules, the alert configurations that are related to the monitoring rules cannot be exported.
If the table that is associated with an imported monitoring rule does not exist in your workspace, the imported monitoring rule is not displayed on the Monitoring Rules page of Data Quality. If you create a table and associate the table with a monitoring rule after the rule is imported, the imported rule is displayed on the rule details page of the table.
If an import package contains monitoring rules and auto triggered nodes, the monitoring rules can be imported only after the auto triggered nodes are imported, committed, and deployed. This way, the auto triggered nodes can be associated with correct monitoring rules.
If an import package contains monitoring rules but does not contain auto triggered nodes, the monitoring rules can be imported. However, the workspace does not contain information about the auto triggered nodes. If an import package contains both the monitoring rules and the auto triggered nodes, the system imports, commits, and deploys the auto triggered nodes, uses the monitoring rules to update the existing monitoring rules, and then associates the auto triggered nodes with the monitoring rules.
Go to the Migration Assistant page
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Development.
Click the
icon in the upper-left corner and choose . The homepage of Migration Assistant appears.
Create a full export task
On the Export Tasks page, click Create Export Task in the upper-right corner.
In the Create Export Task dialog box, configure the parameters.
Parameter
Description
Name
The name of the export task. The name can contain only letters, digits, underscores (_), and periods (.).
Type
The type of the export task. Select Full export for this parameter. In this case, you must configure the Export Scope parameter to specify the types of data objects that you want to export.
Export Scope and Export Object
Valid values of Export Scope:
All Objects: DataWorks exports all types of data objects in your workspace.
Selected Objects: If you select this option, you can select the types of data objects that you want to export from the Export Object drop-down list. For example, you can select Periodic tasks, Resources, and DATA_SERVICE from the Export Object drop-down list.
Export Object Status
The state of the data objects that you want to export. Valid values:
Save: the data objects that are saved
Submit: the data objects that are committed
Deploy: the data objects that are deployed
Blacklist
Specifies whether to enable the blacklist feature for full export based on your business requirements. If you select the Add to Blacklist check box, you can add the nodes and resources that you do not want to export to a blacklist.
Export Version
The DataWorks version. Valid values: Standard, Private Cloud(>=V3.12), and Private Cloud(V3.6.1-V3.11). The DataWorks version determines the format in which data objects are exported. Check the DataWorks version of your workspace before you create an export task.
Remarks
The description of the export task.
Optional. Click Add to Blacklist and run the export task.
If you select the Add to Blacklist check box, perform the following steps to configure the blacklist:
In the Create Export Task dialog box, click Add to Blacklist and then OK.
On the Set Blacklist page, select the data objects that you do not want to export.
Click Add Selected to Blacklist.
Click Export in the upper-right corner.
In the Export confirmation message, click OK.
Optional. If you do not select the Add to Blacklist check box, click Export and then OK in the Create Export Task dialog box.
In the Export Progress dialog box, view the progress of the export task. After the task is completed, click Back to Export Tasks.
Create an incremental export task
On the Export Tasks page, click Create Export Task in the upper-right corner.
In the Create Export Task dialog box, configure the parameters.
Parameter
Description
Name
The name of the export task. The name can contain only letters, digits, underscores (_), and periods (.).
Type
The type of the export task. Select Incremental for this parameter. An incremental export task is used to export all or selected types of data objects that are modified after a specific date. The data objects that you want to export must be saved, committed, or deployed.
Export Scope and Export Object
All Objects: DataWorks exports all types of data objects in your workspace.
Selected Objects: If you select this option, you can select the types of data objects that you want to export from the Export Object drop-down list. For example, you can select Periodic tasks, Resources, and DATA_SERVICE from the Export Object drop-down list.
Export Object Status
The state of the data objects that you want to export. Valid values:
Save: the data objects that are saved
Submit: the data objects that are committed
Deploy: the data objects that are deployed
Modified At
The date on which data objects are modified. The data objects that are generated after this date are incremental data objects.
Export Version
The DataWorks version. Valid values: Standard, Private Cloud(>=V3.12), and Private Cloud(V3.6.1-V3.11).
Remarks
The description of the export task.
Click Export and then OK.
Create a custom export task
On the Export Tasks page, click Create Export Task in the upper-right corner.
In the Create Export Task dialog box, configure the parameters.
Parameter
Description
Name
The name of the export task. The name can contain only letters, digits, underscores (_), and periods (.).
Type
The type of the export task. Select Custom for this parameter. A custom export task is used to export selected types of data objects that are saved, committed, or deployed. The supported types of data objects include Periodic tasks, Manual tasks, Table, and Data source.
Export Version
The DataWorks version. Valid values: Standard, Private Cloud(>=V3.12), and Private Cloud(V3.6.1-V3.11).
Remarks
The description of the export task.
Click Select Export Objects and then OK.
On the Export Objects page, select a type of data object that you want to export from the Export Object drop-down list.
Valid values of Export Object: Table, Periodic tasks, Resources, Manual tasks, Function, Data Quality, DATA_SERVICE, Data source, Components, and Temporary query.
Select the data objects that you want to export and click Add Selected to Export Package.
You can also configure filter conditions such as Export Object, Object Type, and Object Name to search for data objects. Then, click Add All to Export Package to add all the data objects that meet the filter conditions to the package that you want to export.
Click Export in the upper-right corner. In the Export confirmation message, click OK.
View and manage export tasks
On the Export Tasks page, you can view the following information about created export tasks: Name, Type, Created By, Status, Updated At, and Remarks. The operations that you can perform on export tasks vary based on their states.
If an export task is in the Successful state, you can perform the following operations on the task:
Click View Export Report in the Actions column. On the page that appears, view the task information in the Basic Information, Overview, and Details sections.
Click Download Export Package in the upper-right corner to download the package of the export task to a local directory.
NoteThe download link is valid for 30 days. We recommend that you download the package before the validity period ends. After the validity period ends, you need to re-export the project if you want to download the package.
Clone the export task.
Full export task for which the blacklist feature is not enabled: Click Clone in the Actions column. In the Clone dialog box, configure the Name parameter, click Export, and then click OK.
Full export task for which the blacklist feature is enabled: Click Clone in the Actions column. In the Clone dialog box, configure the Name parameter, click Add to Blacklist, and then click OK.
On the Set Blacklist page, select the data objects that you do not want to export and click Add Selected to Blacklist. Then, click Export in the upper-right corner.
Custom export task: Click Clone in the Actions column. In the Clone dialog box, configure the Name parameter, click Select Export Objects, and then click OK.
On the Export Objects page, select the data objects that you want to export and click Add Selected to Export Package. Then, click Export in the upper-right corner.
If an export task is in the Export failed state, you can click View Export Package, Download Export Package, or Re-export in the Actions column based on your business requirements. To retry the export task, click Re-export.
If an export task is a custom export task that is in the Editing state, you can perform the following operations on the task:
Click Edit in the Actions column. On the Export Objects page, modify the data objects that you want to export.
Click View Export Package in the Actions column. On the Export Package Details page, view the task information in the Basic Information, Overview, and Details sections.
Click Delete in the Actions column. In the Delete message, click Ok to delete the task.
If an export task is a full export task that is in the Editing state, you can click Edit, Delete, or View Blacklist in the Actions column based on your business requirements. If you click View Blacklist in the Actions column, you can check the blacklist and click Export to run the export task or click Close in the dialog box that appears.