ApsaraDB for ClickHouse:ApsaraDB for ClickHouse Enterprise Edition

Feature name

Description

References

Support for upgrading to a multi-zone deployment

New feature

You can upgrade an Enterprise Edition cluster from a single zone to a multi-zone deployment. You can also migrate the zones of a multi-zone cluster.

Upgrade to a multi-zone deployment

New Feature

Description

References

New storage types

New feature

Enterprise Edition clusters support ESSD_L1 and ESSD_L2 as storage options.

[Annoucement] Public preview: Disk storage for ApsaraDB for ClickHouse Enterprise Edition

Version

Description

References

Enterprise Edition 24.12

• New features

  • Implemented a primary index cache for MergeTree tables. This cache is enabled by use_primary_key_cache. It supports on-demand loading, similar to a mark cache, and prefetching through prewarm_primary_key_cache.

  • Added Cluster table functions for Iceberg, Delta Lake, and Hudi. Implemented the read support for tables in the Iceberg format, including changes in column order and column names, and simple type extensions.

• Improvement

  Promoted the JSON, Dynamic, and Variant types from experimental to Beta status.

• Performance optimization

  Optimized JOIN performance.

Release details of ApsaraDB for ClickHouse Enterprise Edition V24.12

Version

Description

References

Enterprise Edition 24.10

• New features

  • Refreshable Materialized Views is enabled for Enterprise Edition.

  • The toStartOfInterval function is added to support time alignment with a custom starting point and enhance compatibility with the time bucketing feature of TimescaleDB and PostgreSQL.

  • The arrayZipUnaligned and arrayElementOrNull functions are added to improve Spark compatibility and the flexibility of array operations.

• Fixed issues

  • The following issue is fixed: When optimize_functions_to_subcolumns is enabled, a column type error occurs, and the LowCardinality data type cannot be processed.

  • The following issue is fixed: SQL queries prematurely terminate due to inaccurate memory statistics.

  • The following issue is fixed: When empty fields are used, data in the JSON format cannot be parsed.

  • The following issue is fixed: Query caches cannot be accurately deleted in specific cases.

Release details of ApsaraDB for ClickHouse Enterprise Edition V24.10

Version

Description

References

Enterprise Edition 24.10

New features

The data backup and data restoration features are supported for Enterprise Edition clusters.

• Data backup

• Data restoration

Version

Description

References

Enterprise Edition 24.8

• New features

  • The full_sorting_join algorithm is supported for the ASOF JOIN operation to improve the efficiency of sorting-based JOIN operations.

  • Functions such as percent_rank, changeYear, and changeMonth are added to improve usability.

  • Hive-style partitioning is introduced, which supports various storage engines, such as File, URL, S3, AzureBlobStorage, and HDFS, enabling efficient query and management for large datasets.

• Performance optimization

  • The data access performance of the S3 table engine is improved to speed up queries.

  • The stability of memory allocation is improved.

Release details of ApsaraDB for ClickHouse Enterprise Edition V24.8

Version

Description

References

Enterprise Edition 24.5.1.23258

• New features

  By default, the new analyzer is used by the query engine of the ApsaraDB for ClickHouse cluster.

  Important

  • The new analyzer has improved query performance but may be incompatible with specific SQL statements that are used in the previous versions of Enterprise Edition. This may result in parsing errors.

  • This feature is supported for new clusters that run Enterprise Edition V24.5. If you update the minor engine version of a cluster to V24.5, the cluster still uses the old analyzer.

• Fixed issues

  • The following issue is fixed: When the groupArraySorted function is used, system downtime and memory leaks may occur.

  • The following issue is fixed: When you use skipping indexes, the query results are incorrect.

  • The following issue is fixed: Failures may occur in specific cases.

• Performance optimization

  • The readonly_duration field is added to the system.replicas table to facilitate troubleshooting and impact assessment.

  • The log storage location is changed from cache to S3 to reduce cache usage and improve the cache hit ratio in specific scenarios.

  • The time to clean up inactive data parts is shortened, and the disk usage is optimized in specific cases.

  • By default, consumption starts from the latest offset in the Kafka engine.

  • The default data storage format is changed to Packed, the cache download capability in the background is temporarily disabled, and the usage of the bandwidth and IOPS resources by Object Storage Service (OSS) is optimized.

  • The values of the max_insert_threads and max_final_threads parameters are decreased. The original values of the parameters are excessively large, which led to lock contention between threads and negatively affected performance.

• Stability optimization

  • The kernel compatibility is enabled. Profile-level parameters can be set to the default values specified in the version settings when you create a cluster. This prevents incompatibilities caused by kernel changes during version upgrades and minimizes the impact on business operations.

  • P2P metadata synchronization is enabled, and the values of the operation_timeout_ms and session_timeout_ms parameters are increased to reduce the read and write workloads on the keeper nodes. This prevents cluster startup failures or data write issues caused by excessive workloads.

• Analyzer enabled by default

• The same SQL statement runs without errors on an existing instance but fails on an Enterprise Edition instance of version 24.5 or later. How do I fix this?

• Fix MODIFY COLUMN MODIFY SETTINGS crash for special columns

• Fix LLVM's libunwind in the case when CFA register is RAX

Version

Description

References

Enterprise Edition 24.2.16517

Fixed issues

• The following issue is fixed: The keeper nodes fail to start due to the ZNOAUTH error.

• The following issue is fixed: A failure occurs in the SharedMergeTree engine in rare cases.

• The following issue is fixed: When you use LIMIT and OFFSET in queries on the system.numbers_mt table, the INDEX keyword cannot be used.

N/A

Version

Description

References

Enterprise Edition 24.2.16502

Fixed issues

• The following issue is fixed: The TRUNCATE DATABASE operation interrupts the replication process in a replicated database.

• The following issue is fixed: The part_log redundancy issue occurs in rare cases.

• The following issue is fixed: When you enable the allow_experimental_analyzer=1 setting and use a constant expression in the GROUP BY clause, an error occurs.

• The following issue is fixed: When you query distributed tables, the Cannot convert column because it is non constant in source stream but must be constant in result. error message appears.

• The following issue is fixed: When you use column IS NULL in query conditions, the Unknown identifier and Column is not under aggregate function error messages appear.

N/A

Version

Description

References

Enterprise Edition V24.2.2.16476

• Performance optimization

  • The response interval for resource scaling of serverless clusters is shortened to prevent queries from being interrupted due to delayed scaling.

  • Some parameters are added to ensure that the vertical merge algorithm can be used in some scenarios to improve the merge performance.

  • You can create only table engines that do not store data on local disks when you use the Replicated database engines. This avoids various issues caused by incorrect configurations and operations.

  • The digest_enabled parameter of Keeper is set to 1 to enable real-time data consistency check.

  • The async_replication parameter of Keeper is set to 1 to improve the performance of data replication across multiple nodes when read/write consistency is ensured.

  • The queries per second (QPS) and bandwidth required to access Object Storage Service (OSS) are reduced for the same business load.

• Fixed issues

  • The following issue is fixed: When a session is closed by Keeper, subsequent connections may fail.

  • The following issue is fixed: SQL queries terminate prematurely due to inaccurate memory statistics.

  • The check_not_exists and create_if_not_exists parameters of Keeper are set to 1 to avoid unexpected errors that occur when the ClickHouse server performs operations.

Enterprise Edition V24.2.2.16476

Version

Description

References

Enterprise Edition V24.2.2.16399

• New features

  • The system.dns_cache table is supported to debug Domain Name System (DNS) issues.

  • The mergeTreeIndex, generate_series, and toMillisecond functions are supported.

  • New syntax

    • A syntax is supported to directly grant specific users query and write permissions on views and materialized views without the need for explicit authorization on the underlying tables.

    • ATTACH PARTITION ALL is supported.

    • Enumerations are allowed to be declared in the structure of an external table.

• Changes

  • Ordinary databases cannot be created. Existing Ordinary databases can still be used.

  • SHOW INDEX | INDEXES | INDICES | KEYS no longer sorts results by the primary key columns.

• Optimized feature

  • Stability optimization

    • The ApsaraDB for ClickHouse server and ApsaraDB for ClickHouse Keeper dynamically adjust the memory soft limit based on the compute capacity units (CCUs). This removes the limitation of the small maximum memory supported by Keeper and prevents out-of-memory (OOM) issues to some extent.

    • ApsaraDB for ClickHouse Keeper has added a retry mechanism for failed disk-related operations.

    • ApsaraDB for ClickHouse Keeper stops the startup process if invalid snapshots are detected during service startup. This can prevent data loss.

    • text_log is enabled to help you troubleshoot issues.

  • Performance optimization

    • The size of FileSystemCache can be dynamically adjusted based on the CCUs. In previous versions, the size was fixed to 100 GB.

    • The default compression algorithm changes from LZ4HC(3) to LZ4HC(2), which improves the query efficiency at the expense of less compression.

    • A certain number of logs are cached in the memory to improve the Keeper performance.

  • Serverless optimization

    • The uncompressed cache is disabled to reduce memory usage of the server for elastic scaling.

    • The setting logic of the mark cache size is optimized to reduce the memory usage in some cases for elastic scaling.

    • More aggressive serverless policies are supported. This feature is in invitational preview.

• Fixed issues

  • The following issue is fixed: Queries are unexpectedly terminated due to memory statistics errors.

  • The following issue is fixed: An exception is reported when you execute the first DDL statement after your cluster is restarted.

  • The following issue is fixed: Insertion is interrupted due to the OSS QPS limit.

Release details of ApsaraDB for ClickHouse Enterprise Edition V24.2.2.16399