Queries the information about a new-version synchronization task created in Data Integration. The following types of synchronization tasks are supported: real-time synchronization of all data in a MySQL database to Hologres and batch synchronization of all data in a MySQL database to Hive.
Debugging
Request parameters
Parameter | Type | Required | Example | Description |
Action | String | Yes | GetDIJob | The operation that you want to perform. Set the value to GetDIJob. |
DIJobId | Long | Yes | 11588 | The synchronization task ID. |
WithDetails | Boolean | No | true | Specifies whether to return detailed configuration information of the synchronization task. The detailed configuration information includes TransformationRules, TableMappings, and JobSettings. Valid values: true and false. Default value: true. |
Response parameters
Parameter | Type | Example | Description |
Data | Object | The information about the synchronization task. | |
ProjectId | Long | 22 | The workspace ID. |
DIJobId | Long | 11588 | The synchronization task ID. |
JobName | String | mysql_to_holo_sync_445 | The name of the synchronization task. |
Description | String | Data synchronization from MySQL to Hologres | The description of the synchronization task. |
MigrationType | String | FullAndRealtimeIncremental | The synchronization type. Valid values:
|
SourceDataSourceType | String | MySQL | The type of the source. The value MySQL is returned. |
DestinationDataSourceType | String | Hologres | The type of the destination. Valid values: Hologres and Hive. |
SourceDataSourceSettings | Array of SourceDataSourceSetting | The settings of the source. Only a single source is supported. | |
DataSourceName | String | mysql_datasource_1 | The name of the data source. |
DataSourceProperties | Map | The properties of the data source. | |
String | TimeZone | The properties of the source. The properties of a MySQL data source include TimeZone and Encoding. | |
DestinationDataSourceSettings | Array of DestinationDataSourceSetting | The settings of the destination. Only a single destination is supported. | |
DataSourceName | String | holo_datasource_1 | The name of the data source. |
DataSourceProperties | Map | The properties of the data source. | |
String | TimeZone | The properties of the destination. No properties can be returned because DataWorks does not allow you to configure the properties of a Hologres data source. | |
ResourceSettings | Object | The resource settings. | |
OfflineResourceSettings | Object | The resource used for batch synchronization. | |
ResourceGroupIdentifier | String | S_res_group_111_222 | The identifier of the resource group for Data Integration used for batch synchronization. |
RealtimeResourceSettings | Object | The resource used for real-time synchronization. | |
ResourceGroupIdentifier | String | S_res_group_111_222 | The identifier of the resource group for Data Integration used for real-time synchronization. |
TransformationRules | Array of TransformationRule | The list of transformation rules for objects involved in the synchronization task. Each entry in the list defines a transformation rule. | |
RuleActionType | String | Rename | The type of the action. Valid values:
|
RuleExpression | String | {"expression":"${srcDatasoureName}_${srcDatabaseName}"} | The expression of the rule. The expression is a JSON string. Example of a renaming rule: {"expression":"${srcDatasourceName}_${srcDatabaseName}_0922","variables":[{"variableName":"srcDatabaseName","variableRules":[{"from":"fromdb","to":"todb"}]}]} expression: the expression of the renaming rule. The expression may contain the following variables: ${srcDatasourceName}, ${srcDatabaseName}, and ${srcTableName}. ${srcDatasourceName} indicates the name of the source. ${srcDatabaseName} indicates the name of a source database. ${srcTableName} indicates the name of a source table. variables: the generation rule for a variable used in the expression of the renaming rule. The default value of the specified variable is the original value of the object indicated by the variable. A group of string replacement rules used to change the original values may be returned. variableName: the name of the variable. The variable name is not enclosed in ${}. variableRules: the string replacement rules for variables. The system runs the string replacement rules in sequence for string replacement. from indicates the original string. to indicates the new string. Example of a rule used to add a specific field to the destination and assign a value to the field: {"columns":[{"columnName":"my_add_column","columnValueType":"Constant","columnValue":"123"}]} If no rule of this type is configured, no fields are added to the destination and no values are assigned by default. columnName: the name of the field that is added. columnValueType: the type of the value of the field. Valid values: Constant and Variable. columnValue: the value of the field that is added. If the valueType parameter is set to Constant, a constant of the STRING type is returned for the columnValue parameter. If the valueType parameter is set to Variable, a built-in variable is returned for the columnValue parameter. The following built-in variables are supported: EXECUTE_TIME (LONG data type), DB_NAME_SRC (STRING data type), DATASOURCE_NAME_SRC (STRING data type), TABLE_NAME_SRC (STRING data type), DB_NAME_DEST (STRING data type), DATASOURCE_NAME_DEST (STRING data type), TABLE_NAME_DEST (STRING data type), and DB_NAME_SRC_TRANSED (STRING data type). EXECUTE_TIME indicates the execution time. DB_NAME_SRC indicates the name of a source database. DATASOURCE_NAME_SRC indicates the name of the source. TABLE_NAME_SRC indicates the name of a source table. DB_NAME_DEST indicates the name of a destination database. DATASOURCE_NAME_DEST indicates the name of the destination. TABLE_NAME_DEST indicates the name of a destination table. DB_NAME_SRC_TRANSED indicates the database name obtained after a transformation. Example of a rule used to specify primary key fields for a destination table: {"columns":["ukcolumn1","ukcolumn2"]} If no rule of this type is configured, the primary key fields in the mapped source table are used for the destination table by default. If the table generation method of the destination table is Use Existing Table, Data Integration does not modify the schema of the destination table. If the specified primary key fields do not exist in the destination table, an error is reported when the synchronization task starts to run. If the table generation method of the destination table is Create Table, Data Integration automatically creates the schema of the destination table. The schema contains the primary key fields that you specify. If the specified primary key fields do not exist in the destination table, an error is reported when the synchronization task starts to run. Example of a rule used to process DML messages: {"dmlPolicies":[{"dmlType":"Delete","dmlAction":"Filter","filterCondition":"id > 1"}]} If no rule of this type is configured, the default processing policy for messages generated for insert, update, and delete operations is Normal. dmlType: the DML operation. Valid values: Insert, Update, and Delete. dmlAction: the processing policy for DML messages. Valid values: Normal, Ignore, Filter, and LogicalDelete. Filter indicates conditional processing. The value Filter is returned for the dmlAction parameter only when the dmlType parameter is set to Update or Delete. filterCondition: the condition used to filter DML messages. This parameter is returned only when the dmlAction parameter is set to Filter. |
RuleName | String | rename_rule_1 | The name of the rule. If the values of the RuleActionType parameter and the RuleTargetType parameter are the same for multiple transformation rules, you must make sure that the transformation rule names are unique. |
RuleTargetType | String | Table | The type of the object on which the action is performed. Valid values:
|
TableMappings | Array of TableMapping | The list of mappings between rules used to select synchronization objects in the source and transformation rules applied to the selected synchronization objects. Each entry in the list displays a mapping between a rule used to select synchronization objects and a transformation rule applied to the selected synchronization objects. | |
SourceObjectSelectionRules | Array of SourceObjectSelectionRule | The rule used to select synchronization objects in the source. Multiple rules may be returned. | |
Expression | String | mysql_table_1 | The expression. |
ObjectType | String | Table | The type of the object. Valid values:
|
TransformationRules | Array of TransformationRule | The transformation rules that are applied to the selected synchronization objects. | |
RuleActionType | String | Rename | The type of the action. Valid values:
|
RuleName | String | rename_rule_1 | The name of the rule. If the values of the RuleActionType parameter and the RuleTargetType parameter are the same for multiple transformation rules, you must make sure that the transformation rule names are unique. |
RuleTargetType | String | Table | The type of the object on which the action is performed. Valid values:
|
JobSettings | Object | The settings for the dimension of the synchronization task. The settings include processing policies for DDL messages, policies for data type mappings between source fields and destination fields, and runtime parameters of the synchronization task. | |
ColumnDataTypeSettings | Array of ColumnDataTypeSetting | The settings for data type mappings between source fields and destination fields. The value of this parameter is an array. | |
SourceDataType | String | bigint | The data type of a source field. |
DestinationDataType | String | text | The data type of a destination field. |
DdlHandlingSettings | Array of DdlHandlingSetting | The settings for processing DDL messages. The value of this parameter is an array. | |
Action | String | Critical | The processing policy for DDL messages. Valid values:
|
Type | String | AddColumn | The type of the DDL operation. Valid values:
|
RuntimeSettings | Array of RuntimeSetting | The runtime settings. The value of this parameter is an array. | |
Name | String | runtime.offline.concurrent | The name of the configuration item. Valid values:
|
Value | String | 1 | The value of the configuration item. |
CycleScheduleSettings | Object | The settings for periodic scheduling. | |
CycleMigrationType | String | Full | The synchronization type that requires periodic scheduling. Valid values:
|
ScheduleParameters | String | bizdate=$bizdate | The scheduling parameters. |
ChannelSettings | String | {"structInfo":"MANAGED","storageType":"TEXTFILE","writeMode":"APPEND","partitionColumns":[{"columnName":"pt","columnType":"STRING","comment":""}],"fieldDelimiter":""} | The channel control settings for the synchronization task. The value of this parameter is a JSON string. |
CreatedTime | Long | 1671516776 | The timestamp when the synchronization task was created. The timestamp is accurate to the second. |
CreatedUid | String | 100000001 | The ID of the user who creates the synchronization task. |
UpdatedTime | Long | 1673859985 | The timestamp when the synchronization task was last modified. The timestamp is accurate to the second. |
UpdatedUid | String | 100000001 | The ID of the user who last modifies the synchronization task. |
StartedTime | Long | 1673859999 | The timestamp when the synchronization task was last started. The timestamp is accurate to the second. |
StartedUid | String | 100000001 | The ID of the user who last starts the synchronization task. |
JobStatus | String | Finished | The task status. Valid values:
|
ErrorMessage | String | error details xxx | The error message returned if the value of the JobStatus parameter is Failed. |
RunStats | Map | The information about the running of the synchronization task. | |
String | StructureMigrationTotalTables | The detailed statistics on the running of the synchronization task.
| |
RequestId | String | 0000-ABCD-EFG**** | The request ID. |
Examples
Sample requests
http(s)://[Endpoint]/?Action=GetDIJob
&DIJobId=11588
&WithDetails=true
&<Common request parameters>
Sample success responses
XML
format
HTTP/1.1 200 OK
Content-Type:application/xml
<GetDIJobResponse>
<Data>
<ProjectId>22</ProjectId>
<DIJobId>11588</DIJobId>
<JobName>mysql_to_holo_sync_445</JobName>
<Description>Data synchronization from MySQL to Hologres</Description>
<MigrationType>FullAndRealtimeIncremental</MigrationType>
<SourceDataSourceType>MySQL</SourceDataSourceType>
<DestinationDataSourceType>Hologres</DestinationDataSourceType>
<SourceDataSourceSettings>
<DataSourceName>mysql_datasource_1</DataSourceName>
<DataSourceProperties>
<key>TimeZone</key>
</DataSourceProperties>
</SourceDataSourceSettings>
<DestinationDataSourceSettings>
<DataSourceName>holo_datasource_1</DataSourceName>
<DataSourceProperties>
<key>TimeZone</key>
</DataSourceProperties>
</DestinationDataSourceSettings>
<ResourceSettings>
<OfflineResourceSettings>
<ResourceGroupIdentifier>S_res_group_111_222</ResourceGroupIdentifier>
</OfflineResourceSettings>
<RealtimeResourceSettings>
<ResourceGroupIdentifier>S_res_group_111_222</ResourceGroupIdentifier>
</RealtimeResourceSettings>
</ResourceSettings>
<TransformationRules>
<RuleActionType>Rename</RuleActionType>
<RuleExpression>{"expression":"${srcDatasoureName}_${srcDatabaseName}"}</RuleExpression>
<RuleName>rename_rule_1</RuleName>
<RuleTargetType>Table</RuleTargetType>
</TransformationRules>
<TableMappings>
<SourceObjectSelectionRules>
<Expression>mysql_table_1</Expression>
<ObjectType>Table</ObjectType>
</SourceObjectSelectionRules>
<TransformationRules>
<RuleActionType>Rename</RuleActionType>
<RuleName>rename_rule_1</RuleName>
<RuleTargetType>Table</RuleTargetType>
</TransformationRules>
</TableMappings>
<JobSettings>
<ColumnDataTypeSettings>
<SourceDataType>bigint</SourceDataType>
<DestinationDataType>text</DestinationDataType>
</ColumnDataTypeSettings>
<DdlHandlingSettings>
<Action>Critical</Action>
<Type>AddColumn</Type>
</DdlHandlingSettings>
<RuntimeSettings>
<Name>runtime.offline.concurrent</Name>
<Value>1</Value>
</RuntimeSettings>
</JobSettings>
<CreatedTime>1671516776</CreatedTime>
<CreatedUid>100000001</CreatedUid>
<UpdatedTime>1673859985</UpdatedTime>
<UpdatedUid>100000001</UpdatedUid>
<StartedTime>1673859999</StartedTime>
<StartedUid>100000001</StartedUid>
<JobStatus>Finished</JobStatus>
<ErrorMessage>error details xxx</ErrorMessage>
<RunStats>
<key>StructureMigrationTotalTables</key>
</RunStats>
</Data>
<RequestId>0000-ABCD-EFG****</RequestId>
</GetDIJobResponse>
JSON
format
HTTP/1.1 200 OK
Content-Type:application/json
{
"Data" : {
"ProjectId" : 22,
"DIJobId" : 11588,
"JobName" : "mysql_to_holo_sync_445",
"Description" : "Data synchronization from MySQL to Hologres",
"MigrationType" : "FullAndRealtimeIncremental",
"SourceDataSourceType" : "MySQL",
"DestinationDataSourceType" : "Hologres",
"SourceDataSourceSettings" : {
"DataSourceName" : "mysql_datasource_1",
"DataSourceProperties" : {
"key" : "TimeZone"
}
},
"DestinationDataSourceSettings" : {
"DataSourceName" : "holo_datasource_1",
"DataSourceProperties" : {
"key" : "TimeZone"
}
},
"ResourceSettings" : {
"OfflineResourceSettings" : {
"ResourceGroupIdentifier" : "S_res_group_111_222"
},
"RealtimeResourceSettings" : {
"ResourceGroupIdentifier" : "S_res_group_111_222"
}
},
"TransformationRules" : {
"RuleActionType" : "Rename",
"RuleExpression" : "{\"expression\":\"${srcDatasoureName}_${srcDatabaseName}\"}",
"RuleName" : "rename_rule_1",
"RuleTargetType" : "Table"
},
"TableMappings" : {
"SourceObjectSelectionRules" : {
"Expression" : "mysql_table_1",
"ObjectType" : "Table"
},
"TransformationRules" : {
"RuleActionType" : "Rename",
"RuleName" : "rename_rule_1",
"RuleTargetType" : "Table"
}
},
"JobSettings" : {
"ColumnDataTypeSettings" : {
"SourceDataType" : "bigint",
"DestinationDataType" : "text"
},
"DdlHandlingSettings" : {
"Action" : "Critical",
"Type" : "AddColumn"
},
"RuntimeSettings" : {
"Name" : "runtime.offline.concurrent",
"Value" : 1
}
},
"CreatedTime" : 1671516776,
"CreatedUid" : 100000001,
"UpdatedTime" : 1673859985,
"UpdatedUid" : 100000001,
"StartedTime" : 1673859999,
"StartedUid" : 100000001,
"JobStatus" : "Finished",
"ErrorMessage" : "error details xxx",
"RunStats" : {
"key" : "StructureMigrationTotalTables"
}
},
"RequestId" : "0000-ABCD-EFG****"
}
Error codes
HTTP status code | Error code | Error message | Description |
429 | Throttling.Api | The request for this resource has exceeded your available limit. | The number of requests for the resource has exceeded the upper limit. |
429 | Throttling.System | The DataWorks system is busy. Try again later. | The DataWorks system is busy. Try again later. |
429 | Throttling.User | Your request is too frequent. Try again later. | Excessive requests have been submitted within a short period of time. Try again later. |
500 | InternalError.System | An internal system error occurred. Try again later. | An internal error has occurred. Try again later. |
500 | InternalError.UserId.Missing | An internal system error occurred. Try again later. | An internal error has occurred. Try again later. |
For a list of error codes, see Service error codes.