E-MapReduce:Release notes for EMR V5.3.X

Version

Date

EMR V5.3.1

September 2021

EMR V5.3.0

August 2021

Service

Description

SmartData

SmartData is updated to 3.7.1.

Hue

The issue that Impala cannot be used in a high-security cluster is fixed.

Kudu

Kerberos is supported.

HBase

• The time required to restart HBase in a high-security cluster is reduced.

• The issue that Spark 3.1.1 cannot be integrated with HBase is fixed.

• The Graceful Stop process is optimized.

Service

Description

SmartData

SmartData is updated to 3.7.0.

Spark

The incompatibility issue between Spark and Delta Lake is fixed.

Hive

In JindoFS in block storage mode, the metadata of multiple Hive tables can be optimized at the same time.

By default, this feature is disabled.

Delta Lake

• The partitioning feature of Delta Lake is supported.

• The incompatibility issue between the DESC DETAIL command and Spark 3.1.1 is fixed.

YARN

• Information about app IDs, CPU utilization, and memory usage is added to the RESTful APIs of containers for nodes.

• The issue that the Application Master (AM) logs of an automatically released node cannot be viewed is fixed.

• The issue that a cluster cannot be accessed due to historical state store data is fixed.

• The data of a node that is automatically released based on the decommissioning logic of auto scaling can be deleted.

• The Graceful Decommission logic of auto scaling is optimized. The node on which NodeManager runs is marked deprecated only after the NodeManager process is complete.

ZooKeeper

ZooKeeper is updated to 3.6.3.

Flink

• The SmartData service is added.

• Object Storage Service (OSS) can be accessed in password-free mode when you log on to a Dataflow Flink cluster in SSH mode and submit a job in the cluster.

Impala

The issue that the LIST operation is repeatedly performed on directly deleted OSS partition directories is fixed.

Hue

The UI display exception that occurs when you use Hue together with Oozie is fixed.

Kudu

Kudu is updated to 1.14.0.

Clickhouse

ClickHouse is updated to 21.3.13.9.

Iceberg

The Iceberg service is added. The version is 0.12.0.