Real-time message subscription feature of OpenEvent - DataWorks

The OpenEvent module allows you to subscribe to event messages. You can receive notifications about various change events in DataWorks and respond to the events based on your configurations at the earliest opportunity. For example, you can subscribe to table change events to receive notifications about the changes to core tables in real time. You can also subscribe to task change events to implement custom data monitoring of a dashboard that displays the status of real-time synchronization tasks. This topic describes the types of events for subscription by using OpenEvent and performing the subscription configuration procedure.

Permission description

The Open Platform administrator and tenant administrator have read and write permissions on the developer backend. Other tenant-level roles and workspace-level roles have read-only permissions on the developer backend. For more information, see Manage permissions on global-level services.
Only Alibaba Cloud accounts or RAM users to which the AliyunDataWorksFullAccess policy is attached can be used to configure settings on pages in the DataWorks console. For more information, see Manage permissions on the DataWorks services and the entities in the DataWorks console by using RAM policies.

Limits

The OpenEvent module is in public preview for DataWorks Enterprise Edition users and is available in the following regions: China (Beijing), China (Hangzhou), China (Shenzhen), China (Shanghai), China (Zhangjiakou), China (Chengdu), US (Silicon Valley), US (Virginia), Germany (Frankfurt), Japan (Tokyo), China (Hong Kong), and Singapore.
The OpenEvent module allows you to use EventBridge to subscribe to and consume messages. Therefore, you must activate EventBridge. For more information, see What is EventBridge?
Extension point events are classified into tenant-level events and workspace-level events. You can select one of the preceding types of events when you register an extension. For information about the types of extension point events that are supported by DataWorks, see Supported extension point events.

Supported event types

DataWorks allows you to subscribe to regular events and extension point events.

Regular events: You can subscribe to this type of event.
Extension point events: You can subscribe to this type of event, and use an extension provided by DataWorks to configure custom responses to the event messages that you receive.

The following table describes the types of events to which you can subscribe in Operation Center, DataStudio, and Data Governance.

Application scope	Module	Event category	Event description and type	Example
Workspace	Operation Center	Instance status change events	Description: the event that is triggered by the changes to the status of an instance. Type: regular events. You can subscribe to this type of event to receive notifications about instance status changes in real time.	An instance status change event is triggered when the status of an auto triggered node instance changes from Not running to Running or from Running to Run failed.
		Instance operation events	Description: the event that is triggered when you perform an operation such as freezing or unfreezing an instance in Operation Center. Type: regular events. You can subscribe to this type of event to receive notifications about instance operation events in real time.	An instance operation event is triggered when you perform the following operations on an instance in Operation Center: Unfreeze an instance. Freeze an instance. Terminate an instance. Rerun an instance. Set the instance status to Succeeded.
		Scheduling task status change events	Description: the event that is triggered by the changes to the status of a scheduling task. Type: regular events. You can subscribe to this type of event to receive notifications about scheduling task status changes in real time.	A scheduling task status change event can be triggered in the following scenarios: A scheduling task is sent to Data Quality for data quality check. A scheduling task is checked based on branch conditions. A scheduling task is triggered as scheduled, manually triggered, or an event occurs on a scheduling task. The task status changes from Waiting for Resources to Running. A scheduling task finishes running no matter whether the running succeeds or fails. The task status changes from Running to Successful. A scheduling task is rerun after the scheduling task fails to run. The task status changes from Run Failed to Running. A scheduling task is scheduled to run. The task status changes to Waiting for Scheduling Time. The scheduling time is determined by the DueTime or cycleTime parameter.
		Workflow status change events	Description: the event that is triggered by the changes to the status of a workflow. Type: regular events. You can subscribe to this type of event to receive notifications about workflow status changes in real time.	A workflow status change event is triggered when you backfill data for a node. A workflow status change event is triggered when a manually triggered workflow is run.
		Node change events	Description: the event that is triggered by the changes to a node in Operation Center in the production environment. Types: regular events and extension point events.	Regular events: You can develop an on-premises consumer program to consume this type of event message. This way, you can configure custom responses to the event messages that you receive based on your business requirements. Node addition events Node update events Node deletion events Extension point events: This type of event has the same characteristics as regular events. You can also register your on-premises consumer program as a DataWorks extension to implement custom processing logic and process control for extension point events. Pre-event for node undeployment Pre-event for node freezing Pre-event for node unfreezing Note The nodes refer to the auto triggered nodes that are displayed on the Cycle Task page of Operation Center in the production environment.
		Data backfill events	Description: the event that is triggered when you backfill data for a node in Operation Center in the production environment. Type: extension point events.	Event examples: Backfill data for an auto triggered node. Backfill data for an auto triggered node and the descendant nodes of the node. Backfill data for a large number of nodes. Backfill data for multiple nodes that do not have dependencies with each other at a time in advanced mode.
		Operation Center alert events	Description: the event that is triggered when an exception occurs on a task in Operation Center and the exception meets the conditions in an alert rule that you specified in Operation Center. Type: regular events.	Event examples: A task failed to run. Task execution times out. Resources are insufficient.
	Monitoring and alerting	Alert events	Description: the event that is triggered when an alert rule that you specified in Operation Center is matched. Type: regular events.	Event examples: Baseline alerts: If a node in a baseline does not finish running before the estimated latest completion time, an alert is triggered. Event-based alerts: If a node in a baseline fails to run, an alert is triggered. Rule-based alerts: After you create an alert rule for an isolated node that is identified in a workflow, an alert is triggered.
	DataStudio	File change events	Description: the event that is triggered by the changes to the status of a file in DataStudio, such as a node, resource, or function. Type: extension point events. You can subscribe to this type of event and use an extension of DataWorks to configure custom responses to the event messages that you receive.	Event examples: File commit events: When a node, resource, or function is committed, a file commit event is triggered. File deployment events: When a node, resource, or function is deployed, a file deployment event is triggered. Node execution events: When the code of a node is executed, a node execution event is triggered. File deletion events: When a node, resource, or function is deleted, a file deletion event is triggered.
		Table change events	Description: the event that is triggered by the changes to the status of a table on the Workspace Tables page of DataStudio. Type: extension point events. You can subscribe to this type of event and use an extension of DataWorks to configure custom responses to the event messages that you receive.	Event examples: Table commit events: When a table is committed on the Workspace Tables page of DataStudio, a table commit event is triggered. Table deployment events: When a table is deployed on the Workspace Tables page of DataStudio, a table deployment event is triggered. Note You can deploy tables in the Workspace Tables pane only in a workspace in standard mode. Table deployment events can be triggered only in workspaces that are in standard mode.
		Code review events	Description: the event that is triggered when the code of a node is reviewed for improving quality and maintainability of the node code in DataStudio. Type: regular events.	A code review event is triggered when a team member reviews the code of a node before the node is committed for providing a comment and feedback about node committing.
	Security Center	Approval Center events	Description: the event that is triggered when a request order is created, a request order is processed, or permissions on a table are requested. Type: extension point events. You can subscribe to this type of event to receive notifications about event status changes in real time.	Event examples: Request order creation events: When you create a request order by clicking Apply for Function and Resource Permissions in the upper-right corner of the Permission Management tab on the My Data page in Data Map, a request order creation event is triggered. Request order processing events: When the current member that is assigned the Workspace Administrator role processes a request order for tables, resources, or functions on the To Be Approved subtab of the Permission Management tab on the My Data page in Data Map, a request order processing event is triggered. Table permission application events: When a user applies for the read, write, and modify permissions on a specific table based on the business requirements, a table permission application event is triggered.
	Data Quality	Check result feedback events	Description: the event that is triggered when feedback on a check result obtained based on a monitoring rule is sent in Data Quality. Type: regular events. You can subscribe to this type of event to receive notifications about event status changes in real time.	For example, you enable a monitoring rule that is used to check whether the number of rows in a table exceeds 0 for a specific table in Data Quality. After the rule is triggered and the check is complete, users can provide their feedback on the check result. After the feedback is provided, the system sends the feedback that contains information such as the monitoring rule ID and feedback content to a subscriber. A data quality check result feedback event is triggered when the feedback is sent.
	Data Quality	Check completion events	Description: the event that is triggered when a check that is performed based on a monitoring rule is complete in Data Quality. Type: regular events. You can subscribe to this type of event to receive notifications about event status changes in real time.	For example, you enable a monitoring rule that is used to check whether the number of rows in a table exceeds 0 for a specific table in Data Quality. If the rule is triggered and the check is complete, the system sends the check result to a subscriber. The check result shows whether the data passes the quality check and contains samples that are used. A data quality check completion event is triggered when the check result is sent.
Tenant	DataWorks console	Pre-event for workspace deletion	Description: the event that is triggered when a user deletes a workspace on the Workspaces page based on the business requirements. Type: extension point events.	A pre-event for workspace deletion is triggered before a user deletes a workspace on the Workspaces page. The pre-event for workspace deletion can trigger the related extension to check for a specific type of event in a process and blocks the process if such an event occurs.
	DataWorks console	Post-event for workspace deletion	Description: the event that is triggered when a user deletes a workspace on the Workspaces page based on the business requirements. Type: regular events.	A post-event for workspace deletion is triggered after a user deletes a workspace on the Workspaces page.
	Data Upload and Download	Pre-event for data download	Description: the event that is triggered before data is actually downloaded. The related information about the actual data download can be obtained based on the pre-event. Type: extension point events.	A pre-event for data download is triggered when a user downloads data in Data Upload and Download.

Note

Network latency may delay the delivery of event messages.

Procedure

To use the OpenEvent module, you must complete the following configurations:

In the EventBridge console, activate EventBridge and create a custom bus.
Add an event distribution channel in DataWorks.
Configure the custom bus in the EventBridge console.
Publish events in the EventBridge console. For more information, see Publish an event.

Benefits

The OpenEvent module provides the following benefits:

Low O&M costs:
After you enable the message subscription feature in DataWorks, you can subscribe to important DataWorks events with ease.
Improved message publishing and receiving efficiency:
You need to only configure the parameters for message subscription and make preparations for SDK development. This reduces and simplifies manual operations.
Fulfillment of various needs:
You can subscribe to various types of DataWorks events. This allows you to identify important DataWorks changes and respond to the changes at the earliest opportunity.