DataWorks Data Quality allows you to manage a set of custom rules in a centralized manner and create a self-managed rule template library to configure rules in a more efficient manner. This topic describes how to create and configure a custom rule template.
Prerequisites
DataWorks Enterprise Edition or a more advanced edition is activated.
Background information
You can create a rule template on the Rule Template Library page. After the rule template is created, you can manage and use the template.
Precautions
The changes that are made to a custom rule take effect only for tasks that reference the changed custom rule. Existing tasks for which the original custom rule is applied are not affected.
Create a custom rule template
Go to the Data Quality page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Quality.
In the left-side navigation pane, choose .
Click the
icon and select Create Folder. In the Create Folder dialog box, configure the Name and Location parameters and click OK.
In the left-side directory tree, right-click the created folder and select Create Rule Template.
You can also rename or delete a folder.
In the Create Rule Template dialog box, configure the parameters.
Parameter
Description
Template Name
The name of the custom rule template.
Field
The fields that you want to monitor. You can set this parameter only to Custom SQL.
Sampling Method
The sampling method for the custom rule template. You can set this parameter only to Custom SQL.
Set Flag
The
SETclause of the SQL statement that is used to query the fields that you want to monitor.NoteSeparate multiple statements with commas (,). You do not need to add a semicolon (;) at the end of each statement.
Check type
The threshold type for the custom rule template. Valid values: Numeric type and Fluctuation.
Verification Method
The verification method for the custom rule template. The verification methods that can be selected vary based on the threshold type.
If you set the Check type parameter to Numeric type, you can set this parameter only to Compare with a specified value.
Only the value that is calculated by the COUNT or SUM function can be returned. The return value is compared with a specified value.
If you set the Check type parameter to Fluctuation, the following verification methods are used:
Compare the current value with the average value of the last 7 days
Compare the current value with the average value of the last 30 days
Compare the current value with the value 1 day before
Compare the current value with the value 7 days before
Compare the current value with the value 30 days before
The variance between the current value and the value 7 days before
The variance between the current value and the value 30 days before
Compare with the value 1, 7, and 30 days before
Compare with the value of the previous cycle
Custom SQL
The SQL statement that is used to query the fields to be monitored. You can use ${tableName} to specify a table name.
NoteMake sure that the return value is the value in a row of a column and can be compared with the specified threshold.
Location
The name of the folder in which you want to store the custom rule template.
Click OK.
Manage a rule template
In the Templates pane, you can click the name of a rule template to go to the template details page. On the page that appears, you can view, modify, delete, or copy the rule template.
Action | Description |
View | You can view the parameter settings and logs of the rule template and the rules that use the rule template.
|
Edit | Click Edit in the upper-right corner. In the Edit Rule Template dialog box, modify the parameters as required and click OK. |
Delete | Click Delete in the upper-right corner. In the Delete Template message, click OK. |
Copy | Click Copy in the upper-right corner. In the Clone Rule Template dialog box, configure the Template Name and Location parameters and click OK. |
Apply a custom rule template
When you create a monitoring rule, you can select a custom rule template to create the rule based on the rule template.
Go to the Data Quality page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Quality.
Go to the Table Quality Details page.
In the left-side navigation pane, choose . The Configure by Table page appears.
In the Connection section, select the database where the table for which you want to configure rules resides.
Search for the desired table based on conditions, such as the database type, database, and table name. Then, click the name of the table or click Configure Monitoring Rule in the Actions column of the table to go to the Table Quality Details page.
This page displays all the rules configured for the table. You can quickly filter rules based on whether the rules are associated with a monitor. For rules that are not associated with a monitor, you can specify the running mode for the rules.
On the Rule Management tab of the Table Quality Details page, click Create Rule.
Create a monitoring rule by using a custom rule template.
When you reference a custom rule template, the basic configurations of the template, such as FLAG parameter and SQL, are automatically displayed. You can configure the Rule Name parameter based on your business requirements, and the Monitoring Threshold parameter based on the rule type. For example, you must define a normal threshold and a critical threshold for a numeric rule, and you must define a warning threshold in addition to a normal threshold and a critical threshold for a fluctuation-type rule.
For more information about the parameters, see Create a custom rule template.
Click Determine.
References
You can use a custom rule template when you configure a monitoring rule by table. For more information, see Configure monitoring rules for a single table.