GetDIJob - DataWorks - Alibaba Cloud Documentation Center

Queries the information about a new-version synchronization task created in Data Integration. The following types of synchronization tasks are supported: real-time synchronization of all data in a MySQL database to Hologres.

Debugging

You can run this interface directly in OpenAPI Explorer, saving you the trouble of calculating signatures. After running successfully, OpenAPI Explorer can automatically generate SDK code samples.

Debug

Authorization information

The following table shows the authorization information corresponding to the API. The authorization information can be used in the Action policy element to grant a RAM user or RAM role the permissions to call this API operation. Description:

Operation: the value that you can use in the Action element to specify the operation on a resource.
Access level: the access level of each operation. The levels are read, write, and list.
Resource type: the type of the resource on which you can authorize the RAM user or the RAM role to perform the operation. Take note of the following items:
- The required resource types are displayed in bold characters.
- If the permissions cannot be granted at the resource level, All Resources is used in the Resource type column of the operation.
Condition Key: the condition key that is defined by the cloud service.
Associated operation: other operations that the RAM user or the RAM role must have permissions to perform to complete the operation. To complete the operation, the RAM user or the RAM role must have the permissions to perform the associated operations.

Operation	Access level	Resource type	Condition key	Associated operation
dataworks:*	get	All Resources ``	none	none

Request parameters

Parameter	Type	Required	Description	Example
DIJobId	long	Yes	The ID of the synchronization task.	11588
WithDetails	boolean	No	Specifies whether to return detailed configuration information, including TransformationRules, TableMappings, and JobSettings. Valid values: true and false. Default value: true.	true

Response parameters

Parameter	Type	Description	Example
	object	The response parameters.
Data	object	The information about the synchronization task.
ProjectId	long	The workspace ID.	22
DIJobId	long	The ID of the synchronization task.	11588
JobName	string	The name of the synchronization task.	mysql_to_holo_sync_445
Description	string	The description of the synchronization task.
MigrationType	string	The synchronization type. Valid values: FullAndRealtimeIncremental: one-time full synchronization and real-time incremental synchronization RealtimeIncremental: real-time incremental synchronization Full: one-time full synchronization	FullAndRealtimeIncremental
SourceDataSourceType	string	The type of the source. The value MySQL is returned.	MySQL
DestinationDataSourceType	string	The type of the destination. The value Hologres is returned.	Hologres
SourceDataSourceSettings	array<object>	The settings of the source. Only a single source is supported.
SourceDataSourceSetting	object
DataSourceName	string	The name of the data source.	mysql_datasource_1
DataSourceProperties	object	The properties of the data source.
	string	The properties of the source. The properties of a MySQL data source include TimeZone and Encoding.	TimeZone
DestinationDataSourceSettings	array<object>	The settings of the destination. Only a single destination is supported.
DestinationDataSourceSetting	object
DataSourceName	string	The name of the data source.	holo_datasource_1
DataSourceProperties	object	The properties of the data source.
	string	The properties of the destination. No properties are returned because DataWorks does not allow you to configure properties for a Hologres data source.	TimeZone
ResourceSettings	object	The resource settings.
OfflineResourceSettings	object	The resource used for batch synchronization.
ResourceGroupIdentifier	string	The identifier of the resource group for Data Integration used for batch synchronization.	S_res_group_111_222
RealtimeResourceSettings	object	The resource used for real-time synchronization.
ResourceGroupIdentifier	string	The identifier of the resource group for Data Integration used for real-time synchronization.	S_res_group_111_222
TransformationRules	array<object>	The list of transformation rules for objects involved in the synchronization task. Each entry in the list defines a transformation rule.
TransformationRule	object
RuleActionType	string	The type of the action. Valid values: DefinePrimaryKey Rename AddColumn HandleDml	Rename
RuleExpression	string	The expression of the rule. The expression is a JSON string. Example of a renaming rule: {"expression":"${srcDatasourceName}_${srcDatabaseName}_0922","variables":[{"variableName":"srcDatabaseName","variableRules":[{"from":"fromdb","to":"todb"}]}]} expression: the expression of the renaming rule. The expression may contain the following variables: ${srcDatasourceName}, ${srcDatabaseName}, and ${srcTableName}. ${srcDatasourceName} indicates the name of the source. ${srcDatabaseName} indicates the name of a source database. ${srcTableName} indicates the name of a source table. variables: the generation rule for a variable used in the expression of the renaming rule. The default value of the specified variable is the original value of the object indicated by the variable. A group of string replacement rules used to change the original values may be returned. variableName: the name of the variable. The variable name is not enclosed in ${}. variableRules: the string replacement rules for variables. The system runs the string replacement rules in sequence for string replacement. from indicates the original string. to indicates the new string. Example of a rule used to add a specific field to the destination and assign a value to the field: {"columns":[{"columnName":"my_add_column","columnValueType":"Constant","columnValue":"123"}]} If no rule of this type is configured, no fields are added to the destination and no values are assigned by default. columnName: the name of the field that is added. columnValueType: the value type of the field. Valid values: Constant and Variable. columnValue: the value of the field that is added. If the valueType parameter is set to Constant, the value of the columnValue parameter must be a constant of the STRING type. If the valueType parameter is set to Variable, the value of the columnValue parameter must be a built-in variable. The following built-in variables are supported: EXECUTE_TIME (LONG data type), DB_NAME_SRC (STRING data type), DATASOURCE_NAME_SRC (STRING data type), TABLE_NAME_SRC (STRING data type), DB_NAME_DEST (STRING data type), DATASOURCE_NAME_DEST (STRING data type), TABLE_NAME_DEST (STRING data type), and DB_NAME_SRC_TRANSED (STRING data type). EXECUTE_TIME indicates the execution time. DB_NAME_SRC indicates the name of a source database. DATASOURCE_NAME_SRC indicates the name of the source. TABLE_NAME_SRC indicates the name of a source table. DB_NAME_DEST indicates the name of a destination database. DATASOURCE_NAME_DEST indicates the name of the destination. TABLE_NAME_DEST indicates the name of a destination table. DB_NAME_SRC_TRANSED indicates the database name obtained after a transformation. Example of a rule used to specify primary key fields for a destination table: {"columns":["ukcolumn1","ukcolumn2"]} If no rule of this type is configured, the primary key fields in the mapped source table are used for the destination table by default. If the destination table is an existing table, Data Integration does not modify the schema of the destination table. If the specified primary key fields do not exist in the destination table, an error is reported when the synchronization task starts to run. If the destination table is automatically created by the system, Data Integration automatically creates the schema of the destination table. The schema contains the primary key fields that you specify. If the specified primary key fields do not exist in the destination table, an error is reported when the synchronization task starts to run. Example of a rule used to process DML messages: {"dmlPolicies":[{"dmlType":"Delete","dmlAction":"Filter","filterCondition":"id > 1"}]} If no rule of this type is configured, the default processing policy for messages generated for insert, update, and delete operations is Normal. dmlType: the DML operation. Valid values: Insert, Update, and Delete. dmlAction: the processing policy for DML messages. Valid values: Normal, Ignore, Filter, and LogicalDelete. Filter indicates conditional processing. The value Filter is returned for the dmlAction parameter only when the dmlType parameter is set to Update or Delete. filterCondition: the condition used to filter DML messages. This parameter is returned only when the dmlAction parameter is set to Filter.	{"expression":"${srcDatasoureName}_${srcDatabaseName}"}
RuleName	string	The name of the rule. If the values of the RuleActionType parameter and the RuleTargetType parameter are the same for multiple transformation rules, you must make sure that the transformation rule names are unique.	rename_rule_1
RuleTargetType	string	The type of the object on which the action is performed. Valid values: Table Schema	Table
TableMappings	array<object>	The list of mappings between rules used to select synchronization objects in the source and transformation rules applied to the selected synchronization objects. Each entry in the list displays a mapping between a rule used to select synchronization objects and a transformation rule applied to the selected synchronization objects.
TableMapping	object
SourceObjectSelectionRules	array<object>	The rule used to select synchronization objects in the source. Multiple rules may be returned.
SourceObjectSelectionRule	object
Expression	string	The expression.	mysql_table_1
ObjectType	string	The type of the object. Valid values: Table Database	Table
TransformationRules	array<object>	The transformation rules that are applied to the selected synchronization objects.
TransformationRule	object
RuleActionType	string	The type of the action. Valid values: DefinePrimaryKey Rename AddColumn HandleDml	Rename
RuleName	string	The name of the rule. If the values of the RuleActionType parameter and the RuleTargetType parameter are the same for multiple transformation rules, you must make sure that the transformation rule names are unique.	rename_rule_1
RuleTargetType	string	The type of the object on which the action is performed. Valid values: Table Schema	Table
JobSettings	object	The settings for the dimension of the synchronization task. The settings include processing policies for DDL messages, policies for data type mappings between source fields and destination fields, and runtime parameters of the synchronization task.
ColumnDataTypeSettings	array<object>	The settings for data type mappings between source fields and destination fields. The value of this parameter is an array.
ColumnDataTypeSetting	object
SourceDataType	string	The data type of a source field.	bigint
DestinationDataType	string	The data type of a destination field.	text
DdlHandlingSettings	array<object>	The settings for processing DDL messages. The value of this parameter is an array.
DdlHandlingSetting	object
Action	string	The processing policy for DDL messages. Valid values: Ignore: ignores a DDL message. Critical: reports an error for a DDL message. Normal: normally processes a DDL message.	Critical
Type	string	The type of the DDL operation. Valid values: RenameColumn ModifyColumn CreateTable TruncateTable DropTable DropColumn AddColumn	AddColumn
RuntimeSettings	array<object>	The runtime settings. The value of this parameter is an array.
RuntimeSetting	object
Name	string	The name of the configuration item. Valid values: runtime.offline.speed.limit.mb: indicates the maximum transmission rate that is allowed for a batch synchronization task. This configuration item takes effect only when runtime.offline.speed.limit.enable is set to true. runtime.offline.speed.limit.enable: indicates whether throttling is enabled for a batch synchronization task. dst.offline.connection.max: indicates the maximum number of connections that are allowed for writing data to the destination of a batch synchronization task. runtime.offline.concurrent: indicates the maximum number of parallel threads that are allowed for a batch synchronization task. dst.realtime.connection.max: indicates the maximum number of connections that are allowed for writing data to the destination of a real-time synchronization task. runtime.enable.auto.create.schema: indicates whether schemas are automatically created in the destination of a synchronization task. src.offline.datasource.max.connection: indicates the maximum number of connections that are allowed for reading data from the source of a batch synchronization task. runtime.realtime.concurrent: indicates the maximum number of parallel threads that are allowed for a real-time synchronization task.	runtime.offline.concurrent
Value	string	The value of the configuration item.	1
CreatedTime	long	The timestamp when the synchronization task was created. The timestamp is accurate to the second.	1671516776
CreatedUid	string	The ID of the user who creates the synchronization task.	100000001
UpdatedTime	long	The timestamp when the synchronization task was last modified. The timestamp is accurate to the second.	1673859985
UpdatedUid	string	The ID of the user who last modifies the synchronization task.	100000001
StartedTime	long	The timestamp when the synchronization task was last started. The timestamp is accurate to the second.	1673859999
StartedUid	string	The ID of the user who last starts the synchronization task.	100000001
JobStatus	string	The task status. Valid values: Finished Initialized Stopped Failed Running Stopping	Finished
ErrorMessage	string	The error message returned if the value of the JobStatus parameter is Failed.	error details xxx
RunStats	object	The information about the running of the synchronization task.
	string	The detailed statistics on the running of the synchronization task. StructureMigrationTotalTables: the total number of tables whose schemas need to be migrated. StructureMigrationSuccessTables: the number of tables whose schemas are successfully migrated. StructureMigrationFailedTables: the number of tables whose schemas fail to be migrated. StructureMigrationStatus: the status of schema migration. Valid values: Waiting, Running, Finished, Failed, and Stopped. StructureMigrationErrorMessage: the error message returned for schema migration. FullMigrationTotalTables: the total number of tables whose full data needs to be synchronized. FullMigrationSuccessTables: the number of tables whose full data is successfully synchronized. FullMigrationFailedTables: the number of tables whose full data fails to be synchronized. FullMigrationStatus: the status of full synchronization. FullMigrationErrorMessage: the error message returned for full synchronization. RealtimeMigrationMaxDelay: the latency of real-time synchronization. Unit: milliseconds. RealtimeMigrationStatus: the status of real-time synchronization. RealtimeMigrationErrorMessage: the error message returned for real-time synchronization.	StructureMigrationTotalTables
RequestId	string	The request ID.	0000-ABCD-EFG****

Examples

Sample success responses

JSONformat

{
  "Data": {
    "ProjectId": 22,
    "DIJobId": 11588,
    "JobName": "mysql_to_holo_sync_445",
    "Description": "",
    "MigrationType": "FullAndRealtimeIncremental",
    "SourceDataSourceType": "MySQL",
    "DestinationDataSourceType": "Hologres",
    "SourceDataSourceSettings": [
      {
        "DataSourceName": "mysql_datasource_1",
        "DataSourceProperties": {
          "key": "TimeZone"
        }
      }
    ],
    "DestinationDataSourceSettings": [
      {
        "DataSourceName": "holo_datasource_1",
        "DataSourceProperties": {
          "key": "TimeZone"
        }
      }
    ],
    "ResourceSettings": {
      "OfflineResourceSettings": {
        "ResourceGroupIdentifier": "S_res_group_111_222"
      },
      "RealtimeResourceSettings": {
        "ResourceGroupIdentifier": "S_res_group_111_222"
      },
      "RequestedCu": 0
    },
    "TransformationRules": [
      {
        "RuleActionType": "Rename",
        "RuleExpression": "{\"expression\":\"${srcDatasoureName}_${srcDatabaseName}\"}",
        "RuleName": "rename_rule_1",
        "RuleTargetType": "Table"
      }
    ],
    "TableMappings": [
      {
        "SourceObjectSelectionRules": [
          {
            "Expression": "mysql_table_1",
            "ObjectType": "Table"
          }
        ],
        "TransformationRules": [
          {
            "RuleActionType": "Rename",
            "RuleName": "rename_rule_1",
            "RuleTargetType": "Table"
          }
        ]
      }
    ],
    "JobSettings": {
      "ColumnDataTypeSettings": [
        {
          "SourceDataType": "bigint",
          "DestinationDataType": "text"
        }
      ],
      "DdlHandlingSettings": [
        {
          "Action": "Critical",
          "Type": "AddColumn"
        }
      ],
      "RuntimeSettings": [
        {
          "Name": "runtime.offline.concurrent",
          "Value": "1"
        }
      ],
      "CycleScheduleSettings": {
        "CycleMigrationType": "Full",
        "ScheduleParameters": "bizdate=$bizdate"
      },
      "ChannelSettings": "{\"structInfo\":\"MANAGED\",\"storageType\":\"TEXTFILE\",\"writeMode\":\"APPEND\",\"partitionColumns\":[{\"columnName\":\"pt\",\"columnType\":\"STRING\",\"comment\":\"\"}],\"fieldDelimiter\":\"\"}\n"
    },
    "CreatedTime": 1671516776,
    "CreatedUid": "100000001",
    "UpdatedTime": 1673859985,
    "UpdatedUid": "100000001",
    "StartedTime": 1673859999,
    "StartedUid": "100000001",
    "JobStatus": "Finished",
    "ErrorMessage": "error details xxx",
    "RunStats": {
      "key": "StructureMigrationTotalTables"
    }
  },
  "RequestId": "0000-ABCD-EFG****"
}

Error codes

HTTP status code	Error code	Error message
429	Throttling.Api	The request for this resource has exceeded your available limit.
429	Throttling.System	The DataWorks system is busy. Try again later.
429	Throttling.User	Your request is too frequent. Try again later.
500	InternalError.System	An internal system error occurred. Try again later.
500	InternalError.UserId.Missing	An internal system error occurred. Try again later.

For a list of error codes, visit the Service error codes.

Change history

Change time	Summary of changes	Operation
2024-06-04	The Error code has changed. The response structure of the API has changed	View Change Details
2024-01-18	The Error code has changed. The response structure of the API has changed	View Change Details