All Products
Search
Document Center

E-MapReduce:Release notes for EMR V3.51.X

Last Updated:Dec 09, 2024

This topic describes the release notes for E-MapReduce (EMR) V3.51.X, including the release dates, updates, and release version information.

Release dates

Version

Date

EMR V3.51.3

November 2024

EMR V3.51.2

August 29, 2024

EMR V3.51.1

June 21, 2024

EMR V3.51.0

April 23, 2024

Updates

EMR V3.51.3

Service

Description

JindoSDK

JindoSDK is updated to resolve the issue that causes deadlocks.

EMR V3.51.2

Service

Description

JindoCache

  • JindoCache is updated to 6.5.1.

  • The performance of reading data from and writing data to distributed hash tables is improved.

Spark

  • The issue that partition directories cannot be deleted is fixed.

  • The issue related to the Hive package dependency is fixed. This ensures that the connection between Spark and the Metastore client remains uninterrupted.

Trino

  • The issue that some modified configurations may be unexpectedly restored to original configurations during a scale-out is fixed.

  • Data in the OSS-HDFS service that is deployed in a high-security cluster can be queried.

  • The issue that exceptions occur on Trino after DLF-Auth is enabled is fixed.

Presto

Data in the OSS-HDFS service that is installed in a high-security cluster can be queried.

HDFS

The issue that the memory size of NameNodes and DataNodes cannot be modified is fixed.

HBase-HDFS

YARN

  • Multiple timeline events can be sent by the ResourceManager at a time, which improves the processing capability.

  • The logic issue in processing containers and resources of the ResourceManager is fixed.

ZooKeeper

  • The issue that the memory configuration of a node group cannot be modified is fixed.

  • The log configuration files can be reconstructed.

Impala

The issue that client configurations are unexpectedly modified during an auto scaling activity is fixed.

Ranger

The latest version of JindoSDK is supported, which effectively reduces the CPU load.

Knox

The following issue is fixed: The URL of Knox fails to be accessed when a cluster has only one Master Extend node group.

Kafka

The following issue is fixed: The EMR cluster in which Kafka Connect is deployed fails to be started.

StarRocks

The issue that added BE nodes are not displayed after a scale-out is fixed.

Doris

Doris is updated to 2.1.4.

Paimon

Paimon is updated to 0.9-ali-7.

EMR-HOOK

The lineage information of a MaxCompute table can be parsed.

EMR V3.51.1

Service

Description

Spark

Master-Extend node groups are supported.

Hive

Kyuubi

Paimon

Flink that uses Ververica Runtime (VVR) is replaced with Apache Flink, and Data Lake Formation (DLF) catalogs are supported.

Knox

JDK 8 is used for packaging.

Flink

DLF configurations and dependencies that were removed in EMR V3.51.0 have been restored.

EMR V3.51.0

Service

Description

Spark

Spark 3 is updated to 3.4.2.

Celeborn

Celeborn is updated to 0.4.0.

Doris

Doris is updated to 2.1.0.

StarRocks

  • StarRocks 2 is updated to 2.5.18.

  • StarRocks 3 is updated to 3.2.4.

Delta Lake

Delta Lake is updated to 3.0.0.

Iceberg

Iceberg is updated to 1.5.0.

ZooKeeper

ZooKeeper is updated to 3.8.4.

JindoCache

JindoCache is updated to 6.2.5.

Flink

Flink is updated to 1.17.2.

Release version information

DataLake clusters

Service

Version

Hadoop-Common

2.8.5

HDFS

2.8.5

OSS-HDFS

1.0.0

Hive

2.3.9

Spark 2

2.4.8

Spark 3

3.4.2

YARN

2.8.5

Trino

422

Delta Lake

3.0.0

Hudi

0.14.0

Iceberg

1.5.0

Flume

1.11.0

Kyuubi

1.7.3

Tez

0.10.2

OpenLDAP

2.4.46

Ranger

2.3.0

Ranger-plugin

1.0.0

Sqoop

1.4.7

DLF-Auth

2.0.2

Presto

0.283

ZooKeeper

3.8.4

Knox

1.5.0

Celeborn

0.4.0

JindoCache

6.2.5

Paimon

  • EMR V3.51.2: 0.9-ali-7

  • EMR V3.51.1 and EMR V3.51.0: 0.7-ali-1

OLAP clusters

Service

Version

StarRocks 2

2.5.18

StarRocks 3

3.2.4

Doris

  • EMR V5.17.2: 2.1.0

  • EMR V5.17.1 and EMR V3.51.0: 2.1.4

ClickHouse

23.8.2.7

ZooKeeper

3.8.4

Dataflow clusters

Service

Version

Hadoop-Common

2.8.5

HDFS

2.8.5

OSS-HDFS

1.0.0

YARN

2.8.5

OpenLDAP

2.4.46

Ranger

2.3.0

ZooKeeper

3.8.4

Knox

1.5.0

Flink

1.17.2

Kafka

2.12_2.4.1

Kafka-Manager

3.0.0.6

Paimon

  • EMR V3.51.2: 0.9-ali-7

  • EMR V3.51.1 and EMR V3.51.0: 0.7-ali-1

DataServing clusters

Service

Version

Hadoop-Common

2.8.5

HDFS

2.8.5

OSS-HDFS

1.0.0

OpenLDAP

2.4.46

Ranger

2.3.0

Ranger-plugin

1.0.0

ZooKeeper

3.8.4

Knox

1.5.0

HBase

1.7.1

JindoCache

  • EMR V3.51.2: 6.5.1

  • EMR V3.51.1 and EMR V3.51.0: 6.2.5

Phoenix

4.16.1

Custom clusters

Service

Version

Hadoop-Common

2.8.5

HDFS

2.8.5

OSS-HDFS

1.0.0

Hive

2.3.9

Spark 2

2.4.8

Spark 3

3.4.2

YARN

2.8.5

Trino

422

Delta Lake

3.0.0

Hudi

0.14.0

Iceberg

1.5.0

Flume

1.11.0

Kyuubi

1.7.3

Tez

0.10.2

OpenLDAP

2.4.46

Ranger

2.3.0

Ranger-plugin

1.0.0

Sqoop

1.4.7

DLF-Auth

2.0.2

Presto

0.283

StarRocks 2

2.5.18

StarRocks 3

3.2.4

ZooKeeper

3.8.4

Knox

1.5.0

Celeborn

0.4.0

Flink

1.17.2

HBase

1.7.1

JindoCache

  • EMR V3.51.2: 6.5.1

  • EMR V3.51.1 and EMR V3.51.0: 6.2.5

Kudu

1.16.0

Paimon

  • EMR V3.51.2: 0.9-ali-7

  • EMR V3.51.1 and EMR V3.51.0: 0.7-ali-1

Phoenix

4.16.1