All Products
Search
Document Center

AnalyticDB:Use OSS foreign tables for data lake analysis

Last Updated:Dec 04, 2024

This topic describes how to use Object Storage Service (OSS) foreign tables to access OSS data. OSS foreign tables are developed based on the PostgreSQL Foreign Data Wrapper (FDW) framework to access OSS data for data analysis.

Description

The following table describes the operations that you can perform by using OSS foreign tables.

Operation

Description

Query and analyze OSS data

You can use AnalyticDB for PostgreSQL to query the data of OSS foreign tables and join OSS foreign tables with AnalyticDB for PostgreSQL tables for data analysis.

You can query the data of OSS foreign tables in the same manner as querying AnalyticDB for PostgreSQL table data.

Join OSS foreign tables with AnalyticDB for PostgreSQL tables for data analysis

Use OSS foreign tables for data lake analysis

You can use an OSS foreign table to import OSS data to an AnalyticDB for PostgreSQL table.

Use OSS foreign tables to import and export data

You can use an OSS foreign table to export data from an AnalyticDB for PostgreSQL table to OSS.

Use partitioned OSS foreign tables

The table partitioning feature is supported for OSS foreign tables. If the WHERE clause of a query statement contains a partition column, you can improve query efficiency by scanning less data from OSS.

Comparison between OSS foreign tables and OSS external tables

OSS foreign tables are superior to OSS external tables in terms of performance, functionality, and stability. The following table compares the two types of tables.

Item

OSS Foreign Table

OSS External Table

Data import from OSS or export to OSS

Supported

Supported

Analysis for large amounts of data

Provides better performance than OSS external tables.

May fail to meet the expected performance requirements.

Supported object formats

Supports the following object formats:

  • AVRO

  • ORC

  • PARQUET

  • JSON (GZIP-compressed)

  • JSON Lines (GZIP-compressed)

  • CSV (GZIP- and standard Snappy-compressed)

  • TEXT (GZIP- and standard Snappy-compressed)

Supports the following object formats:

  • CSV (GZIP- and standard Snappy-compressed)

  • TEXT (GZIP- and standard Snappy-compressed)

Partitioned tables

Supported

Not supported

References