DataWorks:Best practices for configuring the settings in DataWorks Open Platform to perform a lockdown

Description of subscription configuration

When you configure the Pattern Content parameter for an event rule in EventBridge, specify the event type to the file committing or deployment event. This way, you can subscribe to file committing or deployment events.

• If you use a workspace in standard mode, set type to dataworks:FileChange:CommitFile.

• If you use a workspace in basic mode, set type to dataworks:FileChange:DeployFile.

Step 1: Configure a custom bus

This section describes the core configuration steps and precautions for configuring a custom bus. For more information about how to enable and configure event message subscription, see Enable event message subscription.

1. Log on to the EventBridge console. In the left-side navigation pane, click Event Buses.

2. In the upper-right corner of the Event Buses page, click Quickly Create to create a custom bus.

   1. In the Event Bus step of the Create Custom Event Bus panel, configure the Name parameter and click Next Step to go to the Event Source step.

   2. Click Skip to skip the configurations in the Event Source, Event Rule, and Event Target steps.

3. In the left-side navigation pane, click Event Buses. On the Event Buses page, find the custom bus that you created and click its name. The Overview page of the custom bus appears.

   • In the left-side navigation pane, click Event Rules. On the page that appears, click Create Rule to create an event rule.

   • In this example, the custom bus is configured to receive file committing event messages and file deployment event messages. The following content provides an example on how to configure a demo and the core parameters for the event rule:

     1. Configure Basic Info: In this step, you must configure the Name parameter.

     2. Configure Event Pattern:

        • Event Source Type: Set it to Custom Event Source.

        • Event Source: Leave this parameter empty.

        • Pattern Content: Configure this parameter in the JSON format. Enter the following content. A workspace in standard mode is used in this example.

          Note

          For an event that occurs in a workspace in basic mode, set this parameter to dataworks:FileChange:DeployFile.

          {
              "source": [
                  "acs.dataworks"
              ],
              "type": [
                  "dataworks:FileChange:CommitFile"
              ]
          }

          • source: the identifier of the service in which an event occurs. Set this parameter to acs.dataworks.

          • type: the type of the event that occurs in the service. Set this parameter to dataworks:FileChange:CommitFile.

        • Event Pattern Debugging: You can change the values of the source and type parameters in this section and then click Test. If the test is successful, click Next Step.

     3. Configure Targets:

        • Service Type: Set it to HTTPS or HTTP. For more information about service types, see Manage event rules.

        • URL: Enter the URL for receiving messages pushed by the custom bus, such as https://Server address:Port number/extensions/consumer.

        • Body: Set it to Complete Event.

        • Network Type: Set it to Internet.

Step 2: Configure an event distribution channel

1. Go to the Open Platform page.

   Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose More > Open Platform. The Developer Backend tab appears.

2. In the left-side navigation pane of the Developer Backend page, click OpenEvent. On the page that appears, click Add Event Distribution Channel. In the Add Event Distribution Channel dialog box, configure the parameters.

   • Workspace for Distribution of Event Messages: Select a workspace.

   • Specify Custom Event Bus in EventBridge for Distribution of Event Messages: Select the event bus that you created in Step 1.

3. After you save the event distribution channel, find the event distribution channel on the OpenEvent page and click Enable in the Actions column.

Step 3: Register and configure an extension

1. Go to the Open Platform page.

   Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose More > Open Platform. The Developer Backend tab appears.

2. In the left-side navigation pane of the Developer Backend page, click Extensions. On the page that appears, click Register Extension. In the Register Extension dialog box, configure the parameters.

   1. Select Deployment Method: In this step, set Select a deployment method for your extension to Deploy Based on Self-managed Service.

   2. Register Extension: In this step, configure the following parameters:

      • Extension Name: Enter a custom name for the extension.

      • Processed Extension Points: Select Pre-event for Node Commit and Pre-event for Node Deployment.

      • Workspace for Testing: Specify a test workspace in which you can test the extension before you submit it.

      Configure other parameters based on the on-screen instructions.

3. After you complete the preceding configurations, click OK to save the registered extension.

4. On the Extensions page, find the extension that you registered and click Submit in the Actions column.

   Note

   • Extensions are reviewed by DataWorks. In most cases, an extension review can be completed within three business days after you submit the extension for review.

   • To test the created extension, you must configure the Workspace for Testing parameter.

5. After the extension passes the review, click Publish in the Actions column to publish and use the extension.

6. Click Manage Extensions. You are navigated to the Extension page in SettingCenter. On the Extension page, find the created extension, and enable the extension in the Enable column.

Sample code

When a file is committed, the code compares the file committing time with the dates on which a lockdown should be performed. Then, OpenAPI allows you to call the UpdateIDEEventResult operation to send the comparison result to DataWorks.

Environment requirement: Java8 and Maven. Maven is a build automation tool for Java.

package com.aliyun.dataworks.demo;

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import com.aliyun.dataworks.config.Constants;
import com.aliyun.dataworks.config.EventCheckEnum;
import com.aliyun.dataworks.config.ExtensionParamProperties;
import com.aliyun.dataworks.services.DataWorksOpenApiClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.dataworks_public.model.v20200518.UpdateIDEEventResultRequest;
import com.aliyuncs.dataworks_public.model.v20200518.UpdateIDEEventResultResponse;
import com.aliyuncs.exceptions.ClientException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.List;

/**
 * @author dataworks demo
 */
@RestController
@RequestMapping("/extensions")
public class ExtensionsController {

    @Autowired(required = false)
    private DataWorksOpenApiClient dataWorksOpenApiClient;

    @Autowired
    private ExtensionParamProperties extensionParamProperties;

    /**
     * Receive event messages that are sent from EventBridge.
     * @param jsonParam
     */
    @PostMapping("/consumer")
    public void consumerEventBridge(@RequestBody String jsonParam){
        JSONObject jsonObj = JSON.parseObject(jsonParam);
        String eventCode = jsonObj.getString(Constants.EVENT_CODE_FILED);
        if(Constants.COMMIT_FILE_EVENT_CODE.equals(eventCode)){
            // Initialize the client.
            IAcsClient client = dataWorksOpenApiClient.createClient();
            // Query the current time.
            SimpleDateFormat sdf = new SimpleDateFormat(Constants.YYYY_MM_DD);
            String now = sdf.format(System.currentTimeMillis());
            // Query information about the festivals and holidays in 2022 and the lockdown periods.
            List<String> holidayList = Arrays.asList(extensionParamProperties.getHolidayList().split(","));
            // Check whether the current time is within a lockdown period.
            boolean isExists = holidayList.stream().anyMatch(day -> day.equals(now));
            // The callback method.
            UpdateIDEEventResultRequest updateIDEEventResultRequest = new UpdateIDEEventResultRequest();
            updateIDEEventResultRequest.setMessageId(jsonObj.getString("id"));
            updateIDEEventResultRequest.setExtensionCode(extensionParamProperties.getExtensionCode());
            // You failed the check because the current time is within a lockdown period.
            if(isExists){
                updateIDEEventResultRequest.setCheckResult(EventCheckEnum.FAIL.getCode());
                updateIDEEventResultRequest.setCheckResultTip("You cannot perform a file commit operation within a lockdown period.");
            }else{
                // You passed the check because the current time is not within a lockdown period.
                updateIDEEventResultRequest.setCheckResult(EventCheckEnum.OK.getCode());
                updateIDEEventResultRequest.setCheckResultTip(EventCheckEnum.OK.getName());
            }
            try {
                // The extension processes the event messages and calls an API operation of DataWorks to send the processing result to DataWorks.
                UpdateIDEEventResultResponse acsResponse = client.getAcsResponse(updateIDEEventResultRequest);
                // The ID of the request. You can troubleshoot errors based on the ID.
                System.out.println("acsResponse:" + acsResponse.getRequestId());
            } catch (ClientException e) {
                // The ID of the request. You can troubleshoot errors based on the ID.
                System.out.println("RequestId:" + e.getRequestId());
                // The status code of an error.
                System.out.println("ErrCode:" + e.getErrCode());
                // The description of an error.
                System.out.println("ErrMsg:" + e.getErrMsg());
            }
        }else{
            System.out.println("Failed to filter out other types of events. Check the parameter configurations.");
        }
    }
}
                        

Sample project deployment

1. Prepare the environment and the project.

   • Environment requirement: Java 8 or later and Maven. Maven is a build automation tool for Java.

   • Download link for the project file: extension-demo-deploycontroller.zip (30 KB).

2. Select a deployment mode.

   • On-premises deployment: After you package the project file into a JAR package, run the java -jar yourapp.jar command on the on-premises server in which Java 8 and Maven are deployed to start the service program.

   • Cloud-based deployment: After you package the project file into a JAR package, upload the package to the related runtime environment, such as Docker containers and cloud servers, for deployment.

3. After you download the project file, enter the project root directory and run the packaging command to package the project file into a JAR package.

   mvn clean package -Dmaven.test.skip=true spring-boot:repackage

4. Execute the JAR package:

   java -jar target/extension-demo-deploycontroller-1.0.jar

The following figure shows the successfully started project.Enter http://localhost:8080/index in the address bar of a browser and press Enter. If "hello world!" is returned, the extension is successfully deployed. You can subscribe to event messages after network connections are established between DataWorks and your extension and between EventBridge and your extension.