Hologres V3.0 and later support external databases. This topic describes how to use an external database to manage tables in external data sources and implement federated queries by using internal tables and external tables.
Background
Internal data refers to the data stored in the standard storage class and Infrequent Access (IA) storage class in Hologres. External data refers to the data stored in MaxCompute and Data Lake Formation (DLF). Hologres is seamlessly integrated with MaxCompute. Hologres allows you to directly read data from and write data to MaxCompute to implement high-speed data exchange. Hologres also allows you to use DLF to manage data in Object Storage Service (OSS) data lakes and use Hologres Query Engine (HQE) to achieve high-performance reads and writes of data in data lakes such as Apache Paimon, Hudi, and Delta Lake.
External databases are globally visible. After you connect to a database in Hologres, you can read data from and write data to an external data source by specifying an external table in the ext_db.ext_schema.ext_table format. You can also connect to an external database to execute SQL statements and configure Grand Unified Configuration (GUC) parameters.
Permissions on external databases are managed by the external data sources, such as MaxCompute and DLF. Hologres only passes identities to external data sources. By default, the Hologres service-linked role is used for identity pass-through. Before you access an external data source, you must use your Alibaba Cloud account or a RAM user with the required permissions to grant permissions to the service-linked role. For more information, see Service-linked role for Hologres. Hologres also allows you to use RAM roles to access external data sources. This way, you can customize policies for accessing external data sources. For more information, see RAM role overview. If you use an account which is not an Alibaba Cloud account or a RAM user, such as a BASIC account, you can execute the CREATE USER MAPPING statement to perform account association and authorization. For more information, see CREATE USER MAPPING.
Feature overview
The external database feature allows you to load metadata from external data sources to Hologres without the need to migrate data. This way, internal and external data can be managed in one system. This implements unified metadata management in the lakehouse architecture. Seamless data connection can be implemented based on real-time analysis and query of external data. For information about how to use the CREATE EXTERNAL DATABASE statement, see CREATE EXTERNAL DATABASE.