Queries monitoring results after the data quality of a data source or a compute engine instance is monitored based on monitoring rules.
Debugging
Request parameters
Parameter | Type | Required | Example | Description |
Action | String | Yes | ListQualityResultsByRule | The operation that you want to perform. |
RuleId | Long | Yes | 152322134 | The monitoring rule ID. You can use the ID and information such as a partition filter expression to perform a joint query. |
StartDate | String | Yes | 2020-09-20 00:00:00 | The beginning of the time range to query. Specify the time in the yyyy-MM-dd HH:mm:ss format. This parameter is used together with the EndDate parameter. The interval between the time specified by this parameter and the time specified by the EndDate parameter cannot exceed 7 days. |
EndDate | String | Yes | 2020-09-22 00:00:00 | The end of the time range to query. Specify the time in the yyyy-MM-dd HH:mm:ss format. This parameter is used together with the StartDate parameter. The interval between the time specified by this parameter and the time specified by the StartDate parameter cannot exceed 7 days. |
PageSize | Integer | Yes | 10 | The number of entries per page. Default value: 10. Maximum value: 100. |
PageNumber | Integer | Yes | 1 | The page number. |
ProjectName | String | Yes | autotest | The name of the compute engine instance or data source for which data quality is monitored. |
ProjectId | Long | No | 12345 | The workspace ID. |
Response parameters
Parameter | Type | Example | Description |
HttpStatusCode | Integer | 200 | The HTTP status code. |
ErrorMessage | String | You have no permission. | The error message. |
RequestId | String | CBA58543-00D4-41**** | The request ID. |
ErrorCode | String | 401 | The error code. |
Success | Boolean | true | Indicates whether the request was successful. |
Data | Object | The data structure of the monitoring results returned. | |
PageNumber | Integer | 1 | The page number. |
PageSize | Integer | 10 | The number of entries per page. Default value: 10. Maximum value: 100. |
TotalCount | Long | 200 | The total number of entries returned. |
RuleChecks | Array of RuleChecks | The monitoring results returned. | |
BlockType | Integer | 1 | The strength of the monitoring rule. The strength of a monitoring rule indicates the importance of the rule. Valid values:
|
WarningThreshold | Float | 0.1 | The threshold for a warning alert. The threshold indicates the deviation of the monitoring result from the expected value. You can customize this threshold based on your business requirements. |
Property | String | type | The field whose data quality is checked based on the monitoring rule. This field is a column in the data source table that is monitored. |
TableName | String | dual | The name of the table that is monitored. |
Comment | String | Description of the monitoring rule | The description of the monitoring rule. |
CheckResultStatus | Integer | 2 | The status of the monitoring result. |
TemplateName | String | Compare Number of Unique Field Values Against Expectation | The name of the monitoring template. |
CheckerName | String | fulx | The name of the checker. |
RuleId | Long | 123421 | The ID of the monitoring rule. |
FixedCheck | Boolean | false | Indicates whether the monitoring is performed based on a fixed value. Valid values:
|
Op | String | > | The comparison operator. |
UpperValue | Float | 22200 | The upper limit of the predicted result. The value of this parameter is automatically generated based on the threshold that you specified. |
ActualExpression | String | ds=20200925 | The partition in the monitored data source table. |
ExternalId | String | 123112232 | The node ID. |
TimeCost | String | 10 | The time that was taken to run the monitoring task. |
Trend | String | up | The trend of the monitoring result. |
ExternalType | String | CWF2 | The type of the scheduling system. Only CWF scheduling systems are supported. |
BizDate | Long | 1600704000000 | The data timestamp. If the monitored business entity is offline data, the value is usually one day before the monitoring is performed. |
CheckResult | Integer | 2 | The monitoring result. |
ResultString | String | ResultString | The string of the monitoring result. |
MatchExpression | String | ds=$[yyyymmdd] | The partition filter expression. |
CheckerType | Integer | 0 | The type of the checker. |
ProjectName | String | autotest | The name of the compute engine instance or data source for which data quality is monitored. |
BeginTime | Long | 1600704000000 | The time when the monitoring started. |
DateType | String | YMD | The scheduling frequency. In most cases, the value of this parameter is YMD. This value indicates year, month, and day. |
CriticalThreshold | Float | 0.6 | The threshold for a critical alert. The threshold indicates the deviation of the monitoring result from the expected value. You can customize this threshold based on your business requirements. If a monitoring rule is a strong rule and the critical threshold is exceeded, a critical alert is reported and tasks that are associated with the rule are blocked from running. |
IsPrediction | Boolean | false | Indicates whether the monitoring result is the same as the predicted result. Valid values:
|
RuleName | String | Test | The name of the monitoring rule. |
CheckerId | Integer | 7 | The checker ID. |
DiscreteCheck | Boolean | true | Indicates whether the monitoring is discrete monitoring. Valid values:
|
EndTime | Long | 1600704000000 | The time when the monitoring ended. |
MethodName | String | max | The method used to collect sample data, such as avg, count, sum, min, max, count_distinct, user_defined, table_count, table_size, table_dt_load_count, table_dt_refuseload_count, null_value, null_value/table_count, (table_count-count_distinct)/table_count, or table_count-count_distinct. |
LowerValue | Float | 2344 | The lower limit of the predicted result. The value of this parameter is automatically generated based on the threshold that you specified. |
EntityId | Long | 14534343 | The ID of the partition filter expression. |
WhereCondition | String | type!='type2' | The filter condition of the monitoring task. |
ExpectValue | Float | 90 | The expected value. |
TemplateId | Integer | 5 | The ID of the monitoring template. |
TaskId | String | 16008552981681a0d6**** | The monitoring task ID. |
Id | Long | 2231123 | The primary key ID. |
ReferenceValue | Array of ReferenceValue | The historical sample values. | |
DiscreteProperty | String | type1 | The values of the sample field that are grouped by using the GROUP BY clause. For example, the values of the Gender field are grouped by using the GROUP BY clause. In this case, the values of DiscreteProperty are Male, Female, and null. |
Value | Float | 20 | The sample value. |
BizDate | String | 1600704000000 | The data timestamp. If the monitored business entity is offline data, the value is usually one day before the monitoring is performed. |
SingleCheckResult | Integer | 2 | The string of the monitoring result. |
Threshold | Float | 0.2 | The threshold. |
SampleValue | Array of SampleValue | The current sample values. | |
DiscreteProperty | String | type2 | The values of the sample field that are grouped by using the GROUP BY clause. For example, the values of the Gender field are grouped by using the GROUP BY clause. In this case, the values of DiscreteProperty are Male, Female, and null. |
BizDate | String | 1600704000000 | The data timestamp. If the monitored business entity is offline data, the value is usually one day before the monitoring is performed. |
Value | Float | 23 | The sample value. |
Open | Boolean | true | Indicates whether the monitoring rule is enabled. |
Examples
Sample requests
http(s)://[Endpoint]/?Action=ListQualityResultsByRule
&EndDate=2020-09-22 00:00:00
&PageNumber=1
&PageSize=10
&ProjectName=autotest
&RuleId=152322134
&StartDate=2020-09-20 00:00:00
&<Common request parameters>
Sample success responses
XML
format
HTTP/1.1 200 OK
Content-Type:application/xml
<ListQualityResultsByRuleResponse>
<RequestId>CBA58543-00D4-41****</RequestId>
<QualityResults>
<TotalCount>200</TotalCount>
<PageSize>10</PageSize>
<PageNumber>1</PageNumber>
<RuleChecks>
<Op>&gt;</Op>
<TaskId>16008552981681a0d6****</TaskId>
<EndTime>1600704000000</EndTime>
<DiscreteCheck>true</DiscreteCheck>
<CheckerName>fulx</CheckerName>
<ExpectValue>90</ExpectValue>
<ProjectName>autotest</ProjectName>
<RuleId>123421</RuleId>
<WarningThreshold>0.1</WarningThreshold>
<WhereCondition>type!='type2'</WhereCondition>
<BeginTime>1600704000000</BeginTime>
<CheckResultStatus>2</CheckResultStatus>
<MatchExpression>ds=$[yyyymmdd]</MatchExpression>
<UpperValue>22200</UpperValue>
<IsPrediction>false</IsPrediction>
<CheckResult>2</CheckResult>
<TemplateName>Compare Number of Unique Field Values Against Expectation</TemplateName>
<TimeCost>10</TimeCost>
<FixedCheck>false</FixedCheck>
<MethodName>max</MethodName>
<ResultString>ResultString</ResultString>
<Property>type</Property>
<DateType>YMD</DateType>
<BizDate>1600704000000</BizDate>
<TemplateId>5</TemplateId>
<RuleName>Name of the monitoring rule</RuleName>
<Comment>Description of the monitoring rule</Comment>
<CriticalThreshold>0.6</CriticalThreshold>
<ExternalId>123112232</ExternalId>
<Trend>up</Trend>
<CheckerType>0</CheckerType>
<ExternalType>CWF2</ExternalType>
<CheckerId>7</CheckerId>
<EntityId>14534343</EntityId>
<TableName>dual</TableName>
<BlockType>1</BlockType>
<Id>2231123</Id>
<ActualExpression>ds=20200925</ActualExpression>
<LowerValue>2344</LowerValue>
</RuleChecks>
<RuleChecks>
<SampleValue>
<SingleCheckResult>2</SingleCheckResult>
<Value>20</Value>
<BizDate>1600704000000</BizDate>
<DiscreteProperty>type1</DiscreteProperty>
<Threshold>0.2</Threshold>
</SampleValue>
<SampleValue>
<Value>23</Value>
<BizDate>1600704000000</BizDate>
<DiscreteProperty>type2</DiscreteProperty>
</SampleValue>
<ReferenceValue>
<SingleCheckResult>2</SingleCheckResult>
<Value>20</Value>
<BizDate>1600704000000</BizDate>
<DiscreteProperty>type1</DiscreteProperty>
<Threshold>0.2</Threshold>
</ReferenceValue>
<ReferenceValue>
<Value>23</Value>
<BizDate>1600704000000</BizDate>
<DiscreteProperty>type2</DiscreteProperty>
</ReferenceValue>
</RuleChecks>
</QualityResults>
</ListQualityResultsByRuleResponse>
JSON
format
HTTP/1.1 200 OK
Content-Type:application/json
{
"RequestId" : "CBA58543-00D4-41****",
"QualityResults" : {
"TotalCount" : 200,
"PageSize" : 10,
"PageNumber" : 1,
"RuleChecks" : [ {
"Op" : "&gt;",
"TaskId" : "16008552981681a0d6****",
"EndTime" : 1600704000000,
"DiscreteCheck" : true,
"CheckerName" : "fulx",
"ExpectValue" : 90,
"ProjectName" : "autotest",
"RuleId" : 123421,
"WarningThreshold" : 0.1,
"WhereCondition" : "type!='type2'",
"BeginTime" : 1600704000000,
"CheckResultStatus" : 2,
"MatchExpression" : "ds=$[yyyymmdd]",
"UpperValue" : 22200,
"IsPrediction" : false,
"CheckResult" : 2,
"TemplateName": "Compare Number of Unique Field Values Against Expectation",
"TimeCost" : 10,
"FixedCheck" : false,
"MethodName" : "max",
"ResultString" : "ResultString",
"Property" : "type",
"DateType" : "YMD",
"BizDate" : 1600704000000,
"TemplateId" : 5,
"RuleName": "Name of the monitoring rule",
"Comment": "Description of the monitoring rule",
"CriticalThreshold" : 0.6,
"ExternalId" : 123112232,
"Trend" : "up",
"CheckerType" : 0,
"ExternalType" : "CWF2",
"CheckerId" : 7,
"EntityId" : 14534343,
"TableName" : "dual",
"BlockType" : 1,
"Id" : 2231123,
"ActualExpression" : "ds=20200925",
"LowerValue" : 2344
}, {
"SampleValue" : [ {
"SingleCheckResult" : 2,
"Value" : 20,
"BizDate" : 1600704000000,
"DiscreteProperty" : "type1",
"Threshold" : 0.2
}, {
"Value" : 23,
"BizDate" : 1600704000000,
"DiscreteProperty" : "type2"
} ],
"ReferenceValue" : [ {
"SingleCheckResult" : 2,
"Value" : 20,
"BizDate" : 1600704000000,
"DiscreteProperty" : "type1",
"Threshold" : 0.2
}, {
"Value" : 23,
"BizDate" : 1600704000000,
"DiscreteProperty" : "type2"
} ]
} ]
}
}
Error codes
For a list of error codes, see Service error codes.