Alibaba Cloud E-MapReduce (EMR) Serverless Spark uses the Apache Spark engine. This topic describes the engine versions and the meaning of each part in an engine version.
Engine versions and meaning of each part in an engine version
The engine version of EMR Serverless Spark is in the esr-** (Spark **, Scala **) format. Spark applications run on a standard Java Virtual Machine (JVM), and all Spark tasks are run by using Java or Scala code.
You can use vector-based technologies and native libraries in the runtime environment provided by the Alibaba Cloud Fusion engine to improve the execution performance of SQL applications. For more information about the Fusion engine, see Fusion engine.
Item | Description |
esr-** | The engine version of EMR Serverless Spark. Example: esr-2.1. |
Spark ** | The engine version of Apache Spark that is used by EMR Serverless Spark, such as Spark 3.3.1. |
Scala ** | The version of the Scala programming language that is supported by the engine version. Example: Scala 2.12. |
The following table describes the available engine versions and supported versions of tables in data lakes.
Engine version | Table version |
esr-4.0.0 (Spark 3.5.2, Scala 2.12) |
|
esr-3.0.1 (Spark 3.4.3, Scala 2.12) esr-3.0.0 (Spark 3.3.1, Scala 2.12) |
|
esr-2.4.0 (Spark 3.3.1, Scala 2.12) |
|
esr-2.2.2 (Spark 3.3.1, Scala 2.12) |
|
esr-2.2 (Spark 3.3.1, Scala 2.12) | |
esr-2.1 (Spark 3.3.1, Scala 2.12) | |
esr-1.0 (Spark 2.4.7, Scala 2.11) | - |