E-MapReduce:Release notes for EMR V3.51.X

Version

Date

EMR V3.51.2

August 29, 2024

EMR V3.51.1

June 21, 2024

EMR V3.51.0

April 23, 2024

Service

Description

JindoCache

• JindoCache is updated to 6.5.1.

• The performance of reading data from and writing data to distributed hash tables is improved.

Spark

• The issue that partition directories cannot be deleted is fixed.

• The issue related to the Hive package dependency is fixed. This ensures that the connection between Spark and the Metastore client remains uninterrupted.

Trino

• The issue that some modified configurations may be unexpectedly restored to original configurations during a scale-out is fixed.

• Data in the OSS-HDFS service that is deployed in a high-security cluster can be queried.

• The issue that exceptions occur on Trino after DLF-Auth is enabled is fixed.

Presto

Data in the OSS-HDFS service that is installed in a high-security cluster can be queried.

HDFS

The issue that the memory size of NameNodes and DataNodes cannot be modified is fixed.

HBase-HDFS

YARN

• Multiple timeline events can be sent by the ResourceManager at a time, which improves the processing capability.

• The logic issue in processing containers and resources of the ResourceManager is fixed.

ZooKeeper

• The issue that the memory configuration of a node group cannot be modified is fixed.

• The log configuration files can be reconstructed.

Impala

The issue that client configurations are unexpectedly modified during an auto scaling activity is fixed.

Ranger

The latest version of JindoSDK is supported, which effectively reduces the CPU load.

Knox

The following issue is fixed: The URL of Knox fails to be accessed when a cluster has only one Master Extend node group.

Kafka

The following issue is fixed: The EMR cluster in which Kafka Connect is deployed fails to be started.

StarRocks

The issue that added BE nodes are not displayed after a scale-out is fixed.

Doris

Doris is updated to 2.1.4.

Paimon

Paimon is updated to 0.9-ali-7.

EMR-HOOK

The lineage information of a MaxCompute table can be parsed.

Service

Description

Spark

Master-Extend node groups are supported.

Hive

Kyuubi

Paimon

Flink that uses Ververica Runtime (VVR) is replaced with Apache Flink, and Data Lake Formation (DLF) catalogs are supported.

Knox

JDK 8 is used for packaging.

Flink

DLF configurations and dependencies that were removed in EMR V3.51.0 have been restored.

Service

Description

Spark

Spark 3 is updated to 3.4.2.

Celeborn

Celeborn is updated to 0.4.0.

Doris

Doris is updated to 2.1.0.

StarRocks

• StarRocks 2 is updated to 2.5.18.

• StarRocks 3 is updated to 3.2.4.

Delta Lake

Delta Lake is updated to 3.0.0.

Iceberg

Iceberg is updated to 1.5.0.

ZooKeeper

ZooKeeper is updated to 3.8.4.

JindoCache

JindoCache is updated to 6.2.5.

Flink

Flink is updated to 1.17.2.