This topic describes the limits of Data Lake Analytics (DLA).
Limits on accounts and metadata
Item | Limit |
|---|---|
Maximum number of virtual clusters that can be purchased by each account | 10 |
Maximum number of data source networks that can be configured for each virtual cluster in the serverless Presto engine of DLA | 1 |
Maximum number of databases that can be created by each account | 100 |
Maximum number of tables that can be stored in each database | 4,096 |
Maximum number of columns in each table | 4,096 |
Maximum number of partitions in each table | 60,000 |
Maximum number of Java Database Connectivity (JDBC) connection requests that can be sent by using a client IP address within 1 minute | 10 |
Limits on the serverless Presto engine
Item | Limit |
|---|---|
Alibaba Cloud services supported by data sources of the external tables that are created in the serverless Presto engine of DLA | OSS ApsaraDB RDS Tablestore PolarDB ApsaraDB for Redis ApsaraDB for MongoDB AnalyticDB for MySQL |
Storage system to which asynchronous query results of the serverless Presto engine are written | OSS |
Limits on the serverless Spark engine
Item | Limit |
|---|---|
Data sources of external tables whose metadata can be accessed by the serverless Spark engine from the metadata service of DLA | OSS |
Spark features that are not supported by the serverless Spark engine of DLA | Spark JDBC (Thrift) SparkR |
Destination system to which resource files of the serverless Spark engine, such as JAR, ZIP, and TAR files, are uploaded | OSS |
Accounts and permissions of the serverless Spark engine | RAM users are required. A RAM user must be associated with a DLA sub-account. |
Limits on data lake management
Item | Limit |
|---|---|
Data sources supported by the metadata discovery feature | OSS Log Service Tablestore |
File formats of OSS data sources supported by the metadata discovery feature | CSV, JSON, Parquet, and ORC |
Directory structure of OSS data sources supported by the metadata discovery feature | Database name/Table name/Partition name |
File sampling method of OSS data sources supported by the metadata discovery feature | The latest and oldest files in the directory are sampled, and the first 1,000 rows of each file are read. |
Whether the historical log data that is shipped from Log Service can be discovered by using the metadata discovery feature | If the partition format is changed during log shipping, the data before the change cannot be discovered. |
Data source network supported by one-click data warehousing or data warehousing based on multi-database merging | VPC |
Data sources supported by one-click data warehousing or data warehousing based on multi-database merging | PolarDB for MySQL ApsaraDB RDS for MySQL ApsaraDB RDS for SQL Server ApsaraDB RDS for PostgreSQL ApsaraDB for MongoDB |
Data sources supported by the lakehouse solution | ApsaraDB RDS for MySQL+Data Transmission Service PolarDB for MySQL+Data Transmission Service |