All Products
Search
Document Center

DataWorks:Overview of the DataWorks console

Last Updated:Dec 24, 2024

The DataWorks console provides a visualized page on which you can manage workspaces, resource groups, and purchased resources and services in a centralized manner. The DataWorks console also provides quick access to various functional modules. You can access a module by performing simple operations based on your business requirements. The functional modules can help you implement data development, data integration, and data O&M, and help you monitor and manage the status and performance of DataService Studio.

Overview

You can view different information and perform different operations on various pages in the DataWorks console. The following table describes the pages.

Page

Description

Overview

On the Overview page, you can view the workspaces that are recently used by the current logon account and modify the configurations of the workspaces. You can also view product updates and news on the Overview page.

Workspaces

On the Workspaces page, you can view all workspaces that are created within the current tenant or create a workspace. You can also manage the workspaces to which the current logon account is added as a member.

Resource Groups

On the Resource Groups page, you can view and manage all resource groups within the current logon account or create a resource group.

Image Management

On the Image Management page, you can view custom images that are created within the current logon account. You can also create a custom image, publish the custom image, and use the custom image in data development.

Purchased Resources and Services

On the Purchased Resources and Services page, you can view and manage the services and resource groups that are purchased within the current logon account.

Entry point page of each module

On the entry point page of each module, you can learn the introduction to the module and the procedure to use the module, and get quick access to the service page of the module in a specific workspace.

Computing Resource

You can view available compute engine types and perform O&M on the compute engine instances of the compute engine types.

Alert-related pages

On the Alert Resources page, you can view the alert resources within the current logon account in a specified region. You can also specify the maximum number of alert notifications that can be sent by text message or phone call per day.

Open Platform

On the Open Platform page, you can use the OpenAPI module, OpenEvent module, and Extensions module to integrate your applications with DataWorks.

Lake and Warehouse Integration (Data Lakehouse)

On the Lake and Warehouse Integration (Data Lakehouse) page, you can build a data management platform that integrates data lakes and data warehouses.

Precautions

  • If you log on to the DataWorks console as a RAM user, you may not be able to view specific features in the DataWorks console, or an error may be reported when you use a feature in the DataWorks console. In this case, you must check whether the RAM user that you use is granted the required permissions. For more information, see Manage permissions on the DataWorks services and the entities in the DataWorks console by using RAM policies.

  • The lists of DataWorks workspaces and resource groups vary based on the selected region. Before you use a DataWorks workspace or a resource group, switch to the required region.

Overview

Log on to the DataWorks console. The Overview page appears. On the Overview page, you can view the key scenarios in which DataWorks is used, workspaces that are frequently used within the current logon account, product updates, and other shortcuts.

Workspaces

In the left-side navigation pane, click Workspace. On the Workspaces page, you can view all workspaces that are created within the current tenant in a specified region, manage the workspaces to which your account is added as a member, or create a workspace.

Note

If you log on to the DataWorks console as a RAM user, you are not allowed to view detailed metric data or perform related management operations in the workspaces to which the RAM user is not added as a member. Instead, you can view only the basic information about these workspaces. To view the details of a workspace as a RAM user, you can add the RAM user to the workspace as a member. For more information, see Add workspace members and assign roles to them.

控制台

Operation

Description

References

Select a region

In the area marked with 1 in the preceding figure, you can select a specific region and view the workspaces displayed on the Workspaces page for this region. The workspaces that can be displayed vary based on the selected region.

-.

Create a workspace

In the area marked with 2 in the preceding figure, you can click the button to create a workspace.

Create and manage workspaces.

View basic information about a workspace

In the area marked with 3 in the preceding figure, you can view basic information about a workspace, such as the workspace mode and workspace administrator.

  • Tags: The Tags column displays the tags that are used to mark and organize your Alibaba Cloud resources. A tag consists of a key-value pair.

  • Mode: The Mode column displays the mode of a workspace. DataWorks supports workspaces in basic mode and workspaces in standard mode. We recommend that you use a workspace in standard mode for data development.

  • Administrator: The Administrator column displays the administrator of a workspace. The workspace administrator can add RAM users to the workspace as members on the Workspace Members tab of the Workspace page in Management Center.

  • Resource Group ID: The Resource Group ID column displays the resource group to which a workspace belongs. The resource group is created in the Resource Management console.

    Important

    The resource group displayed in the Resource Group ID column is used to sort resources that are owned by your Alibaba Cloud account. This simplifies the resource and permission management of your Alibaba Cloud account. A resource group that is created in the Resource Management console is different from a resource group that is used to run tasks in DataWorks. For more information about Alibaba Cloud resource groups, see What is Resource Management?

Perform operations related to a workspace

In the area marked with 4 in the preceding figure, you can view details of the current workspace, click one of the items that are displayed to access the related service, or perform operations related to a workspace.

  • Details: You can click Details to view details of the current workspace, including the running details of instances whose data timestamp is the previous day, issues to be governed, data models, and data metrics.

  • Shortcuts: You can click a shortcut in the Actions column of a workspace to quickly access the related service.

  • Manage: You can click Manage to go to the SettingCenter page and perform operations on the Workspace, Data Sources, Cluster Management, and Extension pages.

  • Add Data Source: You can move the pointer over the More icon and select Add Data Source in the Actions column to go to the Data Sources page. On the Data Sources page, you can add a data source and associate the data source with DataStudio for subsequent code development.

  • Delete Workspace: You can delete a workspace. After a workspace is deleted, the workspace cannot be recovered.

  • Disable Workspace: You can disable a workspace that you no longer require. After a workspace is disabled, the tasks in the workspace are no longer automatically scheduled, but the resources of the compute engine instances still exist. You may still be charged fees for the resources.

Upgrade the edition of DataWorks

In the area marked with 5 in the preceding figure, you can use the edition upgrade feature to upgrade the edition of DataWorks.

  • DataWorks advanced editions provide a wider range of service capabilities. Different editions offer different service capabilities.

  • You can activate an advanced edition of DataWorks in a region. Then, you can use the service capabilities that are provided by the edition in all workspaces in the region.

For more information about the differences between DataWorks editions, see Differences among DataWorks editions.

Purchase exclusive resources

In the area marked with 5 in the preceding figure, you can purchase a serverless resource group. After a serverless resource group is purchased, you can click Resource Group in the left-side navigation pane to view the details of the resource group.

For more information about resource groups, see Overview.

Resource Groups

In the left-side navigation pane, click Resource Group. On the Exclusive Resource Groups tab of the Resource Groups page, you can purchase required resource groups or view the details of and manage the resource groups that you have purchased in a specific region.

image

Operation

Description

References

Create a resource group

In the area marked with 1 in the preceding figure, you can click the button to create a serverless resource group.

For more information about resource groups, see Overview.

View basic information about a resource group

You can view basic information about a resource group, such as the status, expiration time, and resource usage of the resource group.

  • Status of a resource group in the area marked with 2 in the preceding figure

    • Starting: The resource group is being started after it is purchased.

    • Running: The resource group is in a normal state and can be used to run tasks.

    • Updating: The resource group is being modified. The modifications include scale-out and scale-in operations.

      Note

      The modification process may require a period of time.

    • Stopped (Expired): The resource group is stopped due to expiration. A subscription resource group enters this state if the resource group is not renewed upon expiration. A resource group in this state cannot be used. If a resource group is not renewed after the resource group has expired for a specified period of time, the system deletes the record of the resource group.

  • Resource usage of a resource group in the area marked with 3 in the preceding figure: If the resource usage of a resource group is excessively high, the running efficiency of tasks that are run on the resource group may be affected. In this case, you must check the resource usage of specific tasks and resolve the issue of unreasonable resource usage at the earliest opportunity.

Perform basic operations on a resource group

In the area marked with 4 in the preceding figure, you can view the detailed information about a resource group and perform operations on a resource group.

  • Details: You can click Details to view basic information about a resource group in the Basic Information section and view information on the following tabs: Resource Usage, Used Parallel Threads of Data Scheduling, Data Integration, Data Computing, DataService Studio, Data Scheduling, and Individual Development Environment.

  • Network Settings: If you want to connect a resource group to a data source that is deployed in a specific network environment, you must configure network settings for the resource group. Before you configure network settings, you can select a network connection solution and then refer to the related topic to perform the configuration.

  • Associate Workspace: After a resource group is created, you must associate the resource group with a desired workspace. This way, you can use the resource group in the workspace.

    You can change the workspace for a resource group only after you are granted the ModifyResourceGroup permission. For information about how to obtain the permission, see the Use custom policies to manage permissions on the entities in the DataWorks console in a fine-grained manner section of the "Manage permissions on the DataWorks services and the entities in the DataWorks console by using RAM policies" topic.

  • More: You can move the pointer over the image icon and select options in the Actions column to perform operations on a resource group, including scale-out, scale-in, renewal, unsubscription, quota management, and change of the maximum number of parallel threads for data scheduling.

Note

A change operation that is performed on a resource group may require a period of time.

Image Management

In the left-side navigation pane, click Image Management. On the Image Management page, you can view DataWorks official images. If a specific development environment, such as a third-party dependency, is required for tasks that are run on a serverless resource group, you can use the image management feature to create a custom image that integrates required development packages and dependencies. Then, you can specify resources in the serverless resource group as the execution resources for running the tasks and specify the custom image as the runtime environment. For more information, see Manage images.

Purchased Resources and Services

In the left-side navigation pane, click Purchased Resources and Services. On the Purchased Resources and Services page, you can view all subscription and pay-as-you-go resource groups, services, and features that are purchased within the current logon account. You can also view the details of the resource groups and click the entry points provided in the Pay-As-You-Go section to view the related bill and billing rule. In addition, you can renew and unsubscribe from subscription resource groups, services, and features, and upgrade or downgrade the configurations of subscription resource groups, services, and features. For more information, see View spending details and Billing overview.

Entry point page of each module

You can learn about the features and use procedure of each module in the DataWorks console. You can access the following modules in a specified workspace in an efficient manner to perform related operations: Data Integration, Data Modeling, DataStudio, Operation Center, Data Quality, DataAnalysis, Data Map, Security Center, Data Governance Center, DataService Studio, and Management Center.

image

Computing Resource

In the left-side navigation pane, click Computing Resource. Then, click the name of a desired compute engine type to view the list of compute engine instances of this type and perform the related O&M operations on the compute engine instances.

Item

Description

References

MaxCompute

Click MaxCompute to go to the Projects page in the MaxCompute console to configure, view the details of, and manage MaxCompute projects.

Manage projects

Hologres

Click Hologres to go to the Hologres console to manage Hologres instances.

Hologres console

AnalyticDB for MySQL

Click AnalyticDB for MySQL to go to the AnalyticDB for MySQL console and perform operations such as cluster creation, cluster O&M, and data development.

Instructions for new AnalyticDB for MySQL users

AnalyticDB for PostgreSQL

Click AnalyticDB for PostgreSQL to go to the AnalyticDB for PostgreSQL console to manage AnalyticDB for PostgreSQL instances.

Create an instance

Alert-related pages

In the left-side navigation pane, choose Others > Alert Resource or choose Others > Alert Contacts. On the Alert Contacts page, you can view and specify information about an alert contact. On the Alert Resources page, you can specify the maximum number of alert notifications that can be sent by text message or phone call per day.

Page

Description

References

Alert Resources

On the Alert Resources page, you can view the usage of alert resources in the Daily Usage of Alert Resources and Alert Resource Usage in Current Month sections. You can specify the maximum number of alert notifications that can be sent by using each notification method per day.

Important
  • The upper limits that you specify for each notification method take effect only in the current region.

  • If the upper limits are reached, alert notifications can no longer be sent on the same day.

-

Alert Contacts

On the Alert Contacts page, you can specify a RAM user or a RAM role as an alert contact and specify contact information. You can also synchronize the contact information that is configured for RAM users with simple operations.

Note

A configured mobile phone number or email address takes effect only after it is activated.

Configure and view alert contacts

Open Platform

In the left-side navigation pane, choose More > Open Platform. DataWorks Open Platform provides the OpenAPI, OpenEvent, and Extensions modules. You can use the modules to integrate your applications with DataWorks and subscribe to event messages. These modules facilitate process management for data processing, data governance, and data O&M, and allow you to identify important changes in DataWorks and respond to the changes at the earliest opportunity.

Module

Description

References

OpenAPI

The OpenAPI module displays the number of API calls that are made and information about the API operations that are called. You can call DataWorks API operations to use different features of DataWorks and integrate your applications with DataWorks.

OpenAPI

OpenEvent

The OpenEvent module is an open capability that is provided by DataWorks. The OpenEvent module allows you to subscribe to event messages. This way, you can receive notifications about various change events in DataWorks and respond to the events based on your configurations at the earliest opportunity.

OpenEvent overview

Extensions

DataWorks allows you to use the Extensions module to register programs as DataWorks extensions. You can use the Extensions module together with the message subscription feature provided by the OpenEvent module to manage extension point events and processes. You can also perform custom configuration based on your business requirements.

Extensions

Lake and Warehouse Integration (Data Lakehouse)

The lakehouse solution is implemented by integrating MaxCompute with a Hadoop cluster or integrating MaxCompute with Object Storage Service (OSS). The lakehouse solution allows you to build a data management platform that integrates data lakes and data warehouses. The lakehouse solution integrates the flexibility and broad ecosystem compatibility of data lakes with the enterprise-grade deployment capabilities of data warehouses. For more information, see Lakehouse of MaxCompute.