E-MapReduce:Release notes for EMR V4.6.X

Service

Description

SmartData

SmartData is updated to 3.2.0.

For more information, see SmartData 3.2.X.

Spark

• Spark is updated to 2.4.7.

• jQuery is updated to 3.5.1.

• Spark is compatible with Hive to automatically update table and partition sizes.

• Spark metadata and job running information can be sent to DataWorks.

Hive

• Metadata from Alibaba Cloud Data Lake Formation (DLF) in an HCatalog table is supported.

• Hive metadata and job running information can be sent to DataWorks.

Metastore

• The statistics feature for Hive is added.

• Metadata from DLF in an HCatalog table is supported.

• Methods to obtain an STS token are optimized.

HDFS

• jQuery is updated to 3.5.1.

• HDFS is updated to 3.2.1.

YARN

• YARN is updated to 3.2.1.

• jQuery is updated to 3.5.1.

• Configurations of Fair Scheduler are adjusted.

• Timeline Server is optimized.

Zeppelin

Zeppelin is updated to 0.9.0.

OpenLDAP

• The audit feature is added.

• By default, the SSL port 10636 is enabled.

• Presto can be enabled with one click.

Hue

Presto is supported.

EMRHook

• The EMRHook service is added.

• hive-hook is used to send Hive metadata and job running information to DataWorks.

• spark-hook is used to send Spark metadata and job running information to DataWorks.