ListDataQualityRules - DataWorks - Alibaba Cloud Documentation Center

Queries a list of data quality monitoring rules by page.

Operation description

This API operation is available for all DataWorks editions.

Debugging

You can run this interface directly in OpenAPI Explorer, saving you the trouble of calculating signatures. After running successfully, OpenAPI Explorer can automatically generate SDK code samples.

Debug

Authorization information

There is currently no authorization information disclosed in the API.

Request parameters

Parameter	Type	Required	Description	Example
ProjectId	long	No	The DataWorks workspace ID.	10002
DataQualityEvaluationTaskId	long	No	The ID of the data quality monitoring task that is associated with the rule.	10000
TableGuid	string	No	The ID of the table that is limited by the rule in Data Map.	odps.unit_test.tb_unit_test
Name	string	No	The name of the rule. Fuzzy match is supported.	unit_test
PageSize	integer	No	The number of entries per page. Default value: 10. Maximum value: 200.	10
PageNumber	integer	No	The page number. Default value: 1.	1

Response parameters

Parameter	Type	Description	Example
	object	The response parameters.
RequestId	string	The request ID.	691CA452-D37A-4ED0-9441
PagingInfo	object	The pagination information.
PageNumber	integer	The page number.	1
PageSize	integer	The number of entries per page.	10
TotalCount	integer	The total number of entries returned.	294
DataQualityRules	array<object>	The rules.
DataQualityRule	object
Id	long	The rule ID.	22130
Name	string	The rule name.
TenantId	long	The ID of the DataWorks tenant.	100001
ProjectId	long	The DataWorks workspace ID.	100001
Enabled	boolean	Indicates whether the rule is enabled.	true
Severity	string	The strength of the rule. Valid values: Normal High	High
Description	string	The description of the rule. The description can be up to 500 characters in length.	this is a odps _sql task
Target	object	The monitored object of the rule.
Type	string	The type of the monitored object. Valid values: Table	Table
DatabaseType	string	The type of the database to which the table belongs. Valid values: maxcompute emr cdh hologres analyticdb_for_postgresql analyticdb_for_mysql starrocks	maxcompute
TableGuid	string	The ID of the table that is limited by the rule in Data Map.	odps.unit_test.tb_unit_test
PartitionSpec	string	The configuration of the partitioned table.	ds=$[yyyymmdd-1]
TemplateCode	string	The ID of the template used by the rule.	system::user_defined
SamplingConfig	object	The settings for sampling.
Metric	string	The metrics used for sampling. Valid values: Count: the number of rows in the table. Min: the minimum value of the field. Max: the maximum value of the field. Avg: the average value of the field. DistinctCount: the number of unique values of the field after deduplication. DistinctPercent: the percentage of the number of unique values of the field after deduplication to the number of rows in the table. DuplicatedCount: the number of duplicated values in the field. DuplicatedPercent: the percentage of the number of duplicated values of the field to the number of rows in the table. TableSize: the table size. NullValueCount: the number of rows in which the field is set to null. NullValuePercent: the percentage of the number of rows in which the field is set to null to the number of rows in the table. GroupCount: the field value and the number of rows for each field value. CountNotIn: the number of rows in which the field values are different from the referenced values that you specified in the rule. CountDistinctNotIn: the number of unique values that are different from the referenced values that you specified in the rule after deduplication. UserDefinedSql: indicates that the data is sampled by executing custom SQL statements.	Max
MetricParameters	string	The parameters required for sampling.	{ "Columns": [ "id", "name" ] , "SQL": "select count(1) from table;"}
SettingConfig	string	The statements that are used to configure the parameters required for sampling before you execute the sampling statements. The statements can be up to 1,000 characters in length. Only the MaxCompute database is supported.	SET odps.sql.udf.timeout=600s; SET odps.sql.python.version=cp27;
SamplingFilter	string	The statements that are used to filter unnecessary data during sampling. The statements can be up to 16,777,215 characters in length.	id IS NULL
CheckingConfig	object	The check settings for sample data.
Type	string	The threshold calculation method. Valid values: Fixed Fluctation FluctationDiscreate Auto Average Variance	Fixed
ReferencedSamplesFilter	string	The method that is used to query the referenced samples. To obtain some types of thresholds, you need to query reference values. In this example, an expression is used to indicate the query method of referenced samples.	{ "bizdate": [ "-1", "-7", "-1m" ] }
Thresholds	object	The threshold settings.
Expected	object	The expected threshold setting.
Operator	string	The comparison operator. Valid values: > >= < <= != =	>
Value	string	The threshold value.	100.0
Warned	object	The threshold settings for normal alerts.
Operator	string	The comparison operator. Valid values: > >= < <= != =	>
Value	string	The threshold value.	100.0
Critical	object	The threshold settings for critical alerts.
Operator	string	The comparison operator. Valid values: > >= < <= != =	>
Value	string	The threshold value.	100.0
ErrorHandlers	array<object>	The operations that you can perform after the rule-based check fails.
ErrorHandler	object	The operation that you can perform after the rule-based check fails.
Type	string	The type of the operation. Valid values: SaveErrorData	SaveErrorData
ErrorDataFilter	string	The SQL statement that is used to filter failed tasks. If the rule is defined by custom SQL statements, you must specify an SQL statement to filter failed tasks.	SELECT * FROM tb_api_log WHERE id IS NULL

Examples

Sample success responses

JSONformat

{
  "RequestId": "691CA452-D37A-4ED0-9441",
  "PagingInfo": {
    "PageNumber": 1,
    "PageSize": 10,
    "TotalCount": 294,
    "DataQualityRules": [
      {
        "Id": 22130,
        "Name": "",
        "TenantId": 100001,
        "ProjectId": 100001,
        "Enabled": true,
        "Severity": "High",
        "Description": "this is a odps _sql task",
        "Target": {
          "Type": "Table",
          "DatabaseType": "maxcompute",
          "TableGuid": "odps.unit_test.tb_unit_test",
          "PartitionSpec": "ds=$[yyyymmdd-1]"
        },
        "TemplateCode": "system::user_defined",
        "SamplingConfig": {
          "Metric": "Max",
          "MetricParameters": "{ \"Columns\": [ \"id\", \"name\" ] , \"SQL\": \"select count(1) from table;\"}",
          "SettingConfig": "SET odps.sql.udf.timeout=600s; \nSET odps.sql.python.version=cp27;",
          "SamplingFilter": "id IS NULL"
        },
        "CheckingConfig": {
          "Type": "Fixed",
          "ReferencedSamplesFilter": "{ \"bizdate\": [ \"-1\", \"-7\", \"-1m\" ] }",
          "Thresholds": {
            "Expected": {
              "Operator": ">",
              "Value": "100.0"
            },
            "Warned": {
              "Operator": ">",
              "Value": "100.0"
            },
            "Critical": {
              "Operator": ">",
              "Value": "100.0"
            }
          }
        },
        "ErrorHandlers": [
          {
            "Type": "SaveErrorData\n",
            "ErrorDataFilter": "SELECT * FROM tb_api_log WHERE id IS NULL"
          }
        ]
      }
    ]
  }
}

Error codes

For a list of error codes, visit the Service error codes.

Change history

Change time	Summary of changes	Operation
2024-12-04	The response structure of the API has changed	View Change Details
2024-11-06	The response structure of the API has changed	View Change Details