All Products
Search

This Product

    Data Management:Features

    Document Center

    Data Management:Features

    Last Updated:Jan 15, 2024

    Data Management (DMS) is a one-stop data management platform that you can use to manage data throughout the lifecycle of the data. DMS can help you mine value from data in an efficient and secure manner and promote the digital transformation of your enterprise.

    Level-1 feature

    Level-2 feature

    Level-3 feature

    Description

    References

    Homepage

    Left-side navigation pane

    Instance registration

    You can add or import database instances as a database administrator (DBA) or DMS administrator.

    Register an Alibaba Cloud database instance

    Database instance search

    You can search for a database instance by name, database name, endpoint, or database link name.

    N/A

    Instance list

    • The list displays the normal and abnormal database instances within the current tenant. Database instances that are disabled or deleted are not displayed in the list.

    • You can view the basic information about a database instance. The basic information includes the status, the control mode, the source from which the database instance is imported, the type, the region in which the database instance resides, and the type of environment in which the database instance is used.

    • You can right-click a database instance or a database to select the operation that you want to perform.

    N/A

    Top navigation bar

    Alibaba Cloud tickets

    If an exception occurs when you use DMS, you can submit a ticket.

    N/A

    User center

    You can perform multiple operations in the user center. For example, you can add funds to your Alibaba Cloud account, manage renewals, and view orders and bills. You can also activate, renew, and upgrade the services of DMS.

    Help center

    • Documentation: You can obtain guidelines on how to use the DMS console.

    • DingTalk Group: You can join the DingTalk group of DMS to contact technical support.

    Common user

    Language switching

    You can configure the language of the DMS console.

    N/A

    Display settings

    You can configure the skin and interaction interface of the DMS console.

    • Drop-down Menus in Top Navigation Bar: If you enable this feature, when you move the pointer over the name of the level-1 feature, a drop-down list appears. You can click the name of each feature to access each feature.

    • Shortcuts: If you enable this feature, the frequently used features are displayed in the section to the right of Shortcuts. This way, you can easily access the frequently used feature.

    • Show Tabs: If you enable this feature, multiple tabs can be displayed in the DMS console at the same time if you click more than one feature.

    • Skins: You can select the blue skin or the dark skin. By default, the blue skin is selected.

    Customize the layout and style of the DMS console

    User information

    You can view the username, UID, role, tenant ID, and tenant name of your account. You can also configure and modify the notification methods and log off from the DMS console. If your Alibaba Cloud account belongs to multiple tenants, you can switch between these tenants.

    View system roles

    Resource search

    -

    You can search for a database, a database instance, a table, a feature, or a ticket.

    N/A

    Entry points

    Entry point

    You can easily access the asset dashboard, import or create a database instance, and analyze data.

    N/A

    My tickets

    You can easily go to the My Tickets tab.

    N/A

    My iterations

    You can easily go to the DevOps tab to view the iterations that you create or the iterations in which you participate.

    N/A

    Asset list

    Recently accessed assets

    You can easily go to the SQL Console tab of the databases that are recently accessed. You can also view the performance and details of the databases.

    N/A

    Assets on which you have permissions

    You can view all permissions that you have. These permissions include the permissions on databases, tables, sensitive columns, database instances, and rows. You can release permissions that you no longer need and apply for permissions that are about to expire.

    View owned permissions

    Assets that belong to you

    You can easily view the resources that you own and the permissions that each user has on the resources. You can also grant permissions on the resources to a user and revoke permissions on the resources from a user.

    View resource roles

    Assets that you follow

    You can easily access the databases and tables that you follow.

    N/A

    Data assets

    Global search

    -

    You can search for a database, a database instance, a table, a feature, or a ticket.

    N/A

    Asset categories

    -

    You can add database instances, databases, and tables to different categories. This way, DMS administrators, developers, and O&M personnel can manage and use tables in an efficient manner.

    Use the asset category feature

    Instance management

    Instance list

    • Log on to a database instance: You must use the usernames and passwords of databases to log on to database instances that are managed in Flexible Management and Stable Change control modes.

    • Modify the information about a database instance: You can modify the information that you entered when you import or add a database instance. The information includes the control mode, security rules, environment, and DBA.

    • Refresh and synchronize dictionary: DMS collects the most recent metadata information, such as tables, fields, indexes, and programmable objects, on a database instance.

    • Manage permissions: You can grant permissions that allow users to log on to a database instance or view the performance metrics of the database instance. You can also revoke the permissions from the users.

    • Remove a database instance: You can remove a database instance from the database instance list. You can also delete the configurations, such as permission configurations, of the database instance from the DMS console.

    • Disable a database instance: You can disable a database instance in the DMS console. After you disable the database instance, the database instance is not displayed in the database instance list.

    • Use the access control feature: After you enable the access control feature for a database instance, only authorized users can view and use the database instance.

    • Use the secure access proxy feature: The secure access proxy feature reuses various DMS configurations, such as security rules, data permissions, and sensitive fields. This way, you can securely manage the databases of your enterprise in a secure and more comprehensive manner. You can use MySQL and HTTPS to connect to the proxy endpoints that are generated by the secure access proxy feature for database instances. This way, you can access the database instances outside the DMS console in a secure manner.

    • Change data: On the SQL Console tab, you can limit the SQL statements that are executed to change data. For example, you can forbid the execution of DDL statements for a specific period of time.

    • Use the data security center feature: You can use this feature to detect risks on a database instance. The detection items include geo-redundancy, database audit, fine-grained permission control, sensitive data discovery, and data masking.

    Database list

    • View table details: You can view the schema and indexes of a table and perform operations on a table. For example, you can adjust the sensitivity level of a field, change the data owner of a table, or export the statements that are used to create tables.

    • Query data from individual databases: You can execute custom SQL statements on the SQL Console tab and view the execution results.

    • Manage permissions: You can grant permissions on the resources to a user and revoke permissions on the resources from a user.

    • View the details of a database instance: You can view the details of the database instance to which a database belongs.

    • Locate a database instance: You can view the information about the database instance to which a database belongs.

    • Use the control access feature: After you enable the control access feature for a database, only authorized users can view and use the database.

    • Manage database versions: DMS automatically records each database version and generates statements that are used to create tables for each version. You can obtain the differences between two versions and generate scripts to fix the inconsistencies.

    SQL console

    SQL console

    Visual operation section

    You can view all tables, fields, and indexes of the current database. You can also right-click a table in the database to modify the schema of the table, import data into the table, or export data from the table. You can create, view, execute, and manage programmable objects, such as views, stored procedures, functions, triggers, and events.

    GUI of the SQL Console page

    SQL execution section

    You can write, format, execute, and diagnose SQL statements. You can also modify a result set and update the execution plan of SQL statements. This section provides the intelligent SQL completion prompt feature and allows you to add and manage commonly used SQL statements.

    Diagnose SQL statements

    Execution result section

    You can view the execution results and execution history of SQL statements. The execution results are displayed as a table. You can export the result set to an EXCEL or a TEXT file and generate INSERT scripts for the result set. You can also copy and download a single row or multiple rows of the result set.

    N/A

    Table list

    You can view table schemas and indexes and perform operations on tables. For example, you can apply for permissions on tables, apply to be data owners, adjust data owners, view table details, export statements that are used to create tables, export table schemas, archive data, and adjust the sensitivity levels of fields. You can also use a code generator to produce simple code. This way, the code workload of developers is reduced.

    N/A

    Data analysis

    You can build an enterprise-level data warehouse with a few clicks. This way, data from ApsaraDB RDS for MySQL instances can be synchronized to AnalyticDB for MySQL V3.0 clusters in real time. AnalyticDB for MySQL V3.0 provides features such as data analysis, report development,and periodic scheduling to help you fully extract the value of data.

    Create data warehouses

    Database performance analysis

    Database Autonomy Service (DAS) provides the quick diagnostics, real-time performance, real-time session, and storage analysis features. These features help you achieve a unified view of database status and troubleshoot existing issues at the earliest opportunity to ensure service security and stability.

    View the performance details of a database instance

    Metadata synchronization

    You can collect the most recent metadata of a database, such as tables, fields, indexes, and programmable objects. This way, you can manage permissions on tables, fields, and programmable objects based on different security levels.

    Synchronize the data dictionary of a database instance

    Database export

    You can export the data, schemas, and table creation statements of a database.

    Export databases

    Version management

    DMS automatically records each database version in the database version list. You can view the table creation statements for each version. You can also obtain the differences between two versions and generate scripts to fix the inconsistencies. This way, you can manage different database versions in an efficient manner.

    Manage schema versions

    Operation audit

    You can audit information such as SQL queries, tickets, and logon records for a single database. This way, you can troubleshoot database issues in an efficient manner.

    Audit operations

    Risk audit

    You can identify the risks that are related to metadata, sensitive data, and SQL statements to improve the security and stability of databases.

    Generate a risk audit report

    URL sharing

    You can share the URL of the SQL Console tab of a database and the SQL statements on the tab with other users.

    Share the SQLConsole tab or a ticket

    Super SQL mode

    DMS administrators or DBAs can enable this mode. In this mode, all SQL statements that you submit on the SQL Console tab are executed regardless of the security rules. This mode is suitable for O&M and emergency handling.

    Use the super SQL mode

    Topology

    You can view the physical databases that comprise the current logical database.

    N/A

    Cross-database query

    -

    You can query data in databases and tables across database instances in a convenient manner.

    Query data across multiple database instances

    Database development

    Schema change

    Schema design

    You can follow the settings of an R&D process to design schemas that meet the R&D specifications for specific databases and tables. You can customize R&D processes for different lines of business based on your business requirements to ensure the consistency of schemas among multiple environments, such as the development environment, test environment, and production environment.

    Design schemas

    Schema synchronization

    You can compare the schemas of different databases and generate scripts that can be executed in a specific database to fix inconsistencies. You can use this feature to compare and synchronize the schemas of databases in different environments, such as databases in a production environment and a test environment, databases in different test environments, or databases in different production environments.

    Synchronize schemas

    Shadow table synchronization

    DMS can automatically create a shadow table based on the schema of a source table. DMS attaches a prefix or a suffix to the name of the source table to generate the name of the shadow table. You can use this feature for end-to-end stress testing.

    Create shadow tables for synchronization

    Empty database initialization

    You can synchronize the schemas of the source database to an empty database that does not contain tables. This way, you can efficiently synchronize schemas between databases. You can use this feature to synchronize the schemas of databases that are deployed across multiple regions and units.

    Initialize empty databases

    Table consistency repairing

    You can compare the schemas of tables in different environments, such as the test environment and production environment, find the differences between the schemas, and execute scripts in the specified environment to fix the inconsistencies. You can also use the table consistency repairing feature to ensure consistency between logical databases. For example, you can select a table as the baseline table and check whether other table shards have the same schema as the baseline table.

    Repair table inconsistency

    Data change

    Regular data change

    You can execute SQL statements such as INSERT, UPDATE, DELETE, and TRUNCATE statements to update data in databases. You can modify normal data in scenarios such as data initialization, historical data deletion, troubleshooting, and feature testing.

    Perform regular data change

    Lock-free change

    You can use the lock-free change feature that is based on DML to split a single SQL statement into multiple batches for execution in the DMS console. This can better meet the requirements for large-volume data changes, such as historical data deletion and field updates in a full table. This also ensures execution efficiency and reduces the impact on database performance and database space. You can use the lock-free change feature that is based on DDL to change schemas without the need to lock tables. You can use this feature to prevent business interruptions that can be caused by table locking during schema changes. You can also use this feature to prevent latency for data synchronization between primary and secondary database instances. The latency occurs when schemas are changed by using native online DDL operations.

    Overview

    Historical data deletion

    You can periodically delete historical data based on the lockless change feature to prevent an accumulation of historical data. An accumulation of historical data can affect the stability of the production environment.

    Clear historical data

    Programmable objects

    You can change stored procedures. DMS provides a standardized management process to control the change operations.

    Change programmable objects by using stored routines

    Data import

    You can quickly import a large amount of data to databases. This way, the costs of labor and material resources are reduced.

    Import data

    Data export

    SQL result set export

    You can write and execute SQL query statements and export the result sets.

    Export SQL result sets

    Database export

    You can export a database or specific tables. You can also export only the schemas of a database or specific tables or export table data to extract data for analysis.

    Export databases

    SQL review

    SQL review tickets

    DMS reviews the submitted SQL statements based on security rules and provides optimization suggestions. This feature eliminates SQL statements that do not use indexes or do not conform to database development standards. This feature also helps protect against SQL injection attacks. You can customize SQL specifications in security rules. For example, you can specify that a table must have a primary key, the data type of a primary key column must be restricted, or the number of primary key columns must be restricted.

    SQL review

    Environment construction

    Database cloning

    You can replicate data by cloning databases. You can use this feature to synchronize entire databases or initialize databases across different environments, such as the development environment and test environment.

    Database cloning

    Test data generation

    DMS provides powerful algorithm engines that you can use to generate a large amount of test data, such as random values, region names, and virtual IP addresses. This way, the efficiency of preparing test data is greatly improved.

    Generate test data

    Data tracking tickets

    Data tracking

    The data tracking feature can help you find data changes that are made in a specific period of time and export rollback scripts in batches as needed. Then, you need to submit a data change ticket to run the rollback scripts to restore data in a database.

    Data tracking

    R&D space

    DevOps

    You can use the DevOps feature to customize R&D processes and manage the quality of these processes. This way, the R&D processes are effectively implemented, accidental operations are reduced, and data is protected. The DevOps feature allows you to advance an R&D process stage by stage. You can create specific types of tickets in each iterative stage, which facilitates collaborative development and improves the efficiency of R&D processes.

    Manage iterations

    My tickets

    You can view and approve pending tickets, view the tickets that you have submitted and processed, and view ticket status and ticket details.

    Approve tickets

    Data Transmission Service (DTS)

    Data migration

    -

    You can use DTS to migrate data between homogeneous and heterogeneous data sources. This feature is suitable for the following scenarios: data migration to Alibaba Cloud, data migration between instances in Alibaba Cloud, and database splitting and scale-out.

    Overview of data migration solutions

    Change tracking

    -

    You can use DTS to retrieve incremental data from ApsaraDB RDS for MySQL instances, PolarDB for MySQL instances, PolarDB-X instances, self-managed MySQL databases, and self-managed Oracle databases. Then, you can consume the retrieved incremental data based on your business requirements. This feature is suitable for the following scenarios: cache updates, business asynchronous decoupling, real-time synchronization of heterogeneous data, and real-time data synchronization that involves complex extract, transform, load (ETL) operations.

    Overview of change tracking scenarios

    Data integration

    Data synchronization

    You can use DTS to synchronize data between data sources in real time. This feature is ideal for the following scenarios: active geo-redundancy, geo-disaster recovery, zone-disaster recovery, cross-border data synchronization, query load balancing, cloud business intelligence (BI) systems, and real-time data warehousing.

    Overview of data synchronization solutions

    Batch integration

    The batch integration feature provides a low-code development tool that you can use to develop data processing tasks. You can combine various task nodes to form a data flow and configure periodic scheduling to process or synchronize data. You can use this feature to process complex big data in scenarios such as refined enterprise operations, digital marketing, and intelligent recommendation.

    Overview

    Streaming ETL

    You can extract, transform, process, and load streaming data. This feature helps enrich real-time data processing and computing scenarios and promote the digital transformation of enterprises.

    What is ETL?

    Data development

    Task orchestration

    You can use the task orchestration feature to orchestrate various types of tasks and then schedule and run the tasks. You can create a task flow that consists of one or more task nodes. This way, you can schedule tasks in complex scenarios and improve the efficiency of data development.

    Overview

    Data warehouse development

    You can use the data warehouse development feature to manage projects by workspace, business scenario, and task flow. You can also use this feature to manage data warehouses by layer. This effectively clarifies the data warehouse management requirements in specific scenarios and can fully meet various complex requirements for data warehouse development and management.

    Create a workspace

    Heterogeneous database migration (ADAM)

    Database evaluation

    The database evaluation feature performs intelligent evaluation and analysis based on the data that is collected from the source database. Then, the feature generates a report that contains multiple factors, such as database migration solution, compatibility of the destination database, migration risks, application transformation suggestions, and migration costs.

    Overview

    Database transformation and migration

    This feature provides intelligent tools for database transformation and migration. This facilitates source database comparison, and schema migration and modification based on analysis results.

    Database transformation and migration

    Application evaluation and transformation

    This feature provides application transformation items to migrate databases, such as focus areas and SQL statements, analyzes application frameworks, and provides architecture migration guidelines. This way, you can transform your application in an efficient manner.

    Overview

    Database cutover

    This feature allows you to migrate your service to a cloud database after the database evaluation, database transformation and migration, and application evaluation and transformation are complete.

    Create a cutover project

    Migration lab

    • The periodic SQL collection feature periodically collects SQL statements from a database and automatically combines the SQL statements.

    • The SQL adapter can convert incompatible SQL statements that are migrated from an Oracle database to a PolarDB for PostgreSQL (Compatible with Oracle) cluster in real time. The SQL adapter asynchronously records all SQL statements that need to be modified.

    SQL conversion

    This feature converts the SQL statements of the source database into the SQL statements that can be used in the destination database. This way, the coding workload of developers during database migration is reduced.

    Convert SQL scripts

    Security and compliance

    Permissions

    -

    A tenant can apply for specific operation permissions on a specific database instance, database, table, column, or row. After the application is approved, the tenant can perform operations that are allowed in the permissions. The operation permissions include the query, change, and export permissions. The control permissions include logon permissions on database instances, performance view permissions, database permissions, table permissions, sensitive column permissions, data row permissions, and programmable object permissions.

    Overview

    Security rules

    -

    Security rules are a collection of rules that can be used to control databases in a fine-grained manner. When you query, export, or change data in the DMS console, security rules are used to regulate your behavior. This helps create database operation specifications and R&D processes.

    • Security rules integrate R&D processes, R&D specifications, and approval processes. You can use security rules to coordinate DMS features and allow multiple online developers to collaboratively manage databases.

    • Security rules support a variety of SQL engines. You can customize security rules to check and manage SQL statements.

    • Security rules provide a powerful approval feature. You can customize approval processes based on different user behavior.

    Manage security rules

    Approval processes

    -

    Approval processes are associated with security rules. You can select or configure different approval processes for different user behaviors. A single approval node constitutes a stage. DMS allows you to add, edit, and delete approval nodes. You can add one or more approvers to an approval node.

    Configure approval processes

    IP address whitelists

    -

    IP address whitelists allow you to effectively control the range of users who can access DMS. You can use IP address whitelists to allow only the users from specific trusted network environments to access DMS.

    Access IP whitelist

    Operation audit

    -

    You can query the SQL statements that were executed on the SQL Console tab, tickets, logon records, and operation logs. This way, you can easily troubleshoot database issues and collect data for operation audit.

    Audit operations

    Sensitive data management

    Sensitive data assets

    • You can use this feature to effectively detect and protect sensitive data assets in your enterprise. This prevents sensitive data from being abused or leaked.

    • You can collect sensitive data and enable or disable the sensitive data protection feature for database instances.

    • You can adjust the sensitivity level and data masking permissions on specific fields to a user and revoke permissions on the fields from a user.

    Rule configurations

    • You can use identification templates to scan the fields of databases and tables in an instance by category. This way, you can check whether sensitive fields that meet the specified rules exist. You can also protect high-level sensitive fields.

    • DMS has built-in sensitive data detection rules that you can use to ensure compliance with various laws and regulations. If the built-in detection model cannot meet your business requirements, you can customize a sensitive data detection model with dedicated sensitive data detection capabilities.

    • This feature provides a built-in data masking rule that masks the entire value of a field. You can create custom data masking rules based on the built-in data masking algorithms.

    Sensitive data audit

    This feature records the usage of sensitive data in DMS. This helps you trace the abnormal usage and data breaches of sensitive data.

    Audit sensitive data

    Data watermarks

    -

    This feature allows you to embed watermarks into data or files and extract watermarks from files. This feature is used to trace the source of data during dissemination or distribution.

    Use data watermarks

    Solutions

    Data archiving

    -

    As the amount of data in your database increases, table queries may slow down and your business may be affected. This feature allows you to archive data of large tables from one database to another database at the scheduled time. You can also use this feature to delete archived data from large tables and recycle the storage that is occupied by large tables.

    Data archiving

    Data analysis

    -

    This feature allows you to create a real-time data warehouse with a few clicks. You can synchronize data to an AnalyticDB for MySQL database within seconds. This helps you obtain accurate and up-to-date information about your business and facilitates business analysis and decision-making. This way, the performance of the business is improved.

    Create data warehouses

    Database migration

    -

    This feature allows you to encapsulate service solutions to migrate databases, perform checks, and delete the migrated source databases. In addition, you can change the names of destination databases, precheck whether database migration requirements are met before migration, check whether the data in the source and destination databases is consistent after migration, and delete the migrated source databases. This ensures the integrity of database migration throughout the lifecycle.

    Database migration

    O&M

    User management

    -

    You can add and remove DMS users, manage user permissions, and grant and revoke permissions on database instances, databases, tables, rows, and sensitive columns.

    Manage users

    Task management

    -

    You can create SQL tasks and manage existing tasks. In DMS, tasks are generated for SQL statements that are executed in various tickets such as normal data change, schema design, and database or table synchronization. The SQL statements that are executed on the SQL Console tab are not included.

    Manage tasks

    Configuration management

    -

    DMS administrators and DBAs can modify system configurations based on their business requirements. For example, they can enable automatic synchronization of database instances in the cloud, enable IP address whitelists for access control, and specify whether to allow DMS administrators and DBAs to act beyond their authority to approve or reject tickets.

    Configuration Management

    Notification management

    -

    DMS provides the notification feature that you can use to send notifications about the status updates of tickets and task flows. You can specify notification recipients based on your business requirements. This way, the recipients can receive notifications at the earliest opportunity.

    Manage notification rules

    Database grouping

    -

    You can add databases that run the same engine and are deployed in the same environment to a group. After you create a database group, you can load all databases in the group during data changes or schema design. After you submit the ticket, the changes can take effect in all databases in the group.

    Database grouping

    A hyphen (-) indicates that no features are involved.