By default, AnalyticDB for MySQL Data Lakehouse Edition (V3.0) runs Spark SQL jobs in batch mode. If you want to use job resource groups to run interactive jobs, you must start a Thrift server. After you start a Thrift server, you can run multiple interactive jobs in the same resource group to improve query efficiency. This topic describes how to start and stop a Thrift server for job resource groups in AnalyticDB for MySQL Data Lakehouse Edition (V3.0).
Usage notes
You can start only a single Thrift server for each job resource group.
If a job resource group that has a Thrift server started is used, Spark SQL jobs are run in interactive mode.
If you stop a Thrift server, all the executing and queuing SQL queries are interrupted. Before you stop a Thrift server, make sure that no SQL queries are being executed and queued or that your business is not affected by such interruption.
Start a Thrift server
Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. On the Data Lakehouse Edition tab, find the cluster that you want to manage and click the cluster ID.
In the left-side navigation pane, choose Cluster Management > Resource Management. On the page that appears, click the Resource Groups tab.
On the Resource Groups tab, find the job resource group that you want to manage and click Configure Thrift Server in the Actions column.
In the Configure Thrift Server panel, configure the parameters that are described in the following table.
Parameter
Required
Description
Corresponding configuration parameter
Minimum Executors
Yes
The minimum number of executors allowed for elastic Spark settings. The default value is 0, which specifies that all executors are released if no Spark SQL queries are executed.
Each executor consumes two AnalyticDB compute units (ACUs).
spark.dynamicAllocation.minExecutors
Maximum Executors
Yes
The maximum number of executors allowed for elastic Spark settings. It specifies the maximum executors that are allowed when Spark SQL queries are executed. The minimum value is 1.
Each executor consumes two ACUs. If you specify a large value for this parameter and the computing resources of this job resource group are insufficient, an error may occur. For example, if the maximum computing resources of a job resource group are 16 ACUs, you can set the maximum number of executors to 8.
spark.dynamicAllocation.maxExecutors
JARS
No
The Object Storage Service (OSS) URLs of third-party JAR packages required for the execution environment.
If SQL execution requires JAR packages, you must upload the JAR packages to an OSS bucket first. For information about how to upload objects, see Upload objects.
spark.jars
Maximum Connections
No
The maximum number of sessions allowed for Spark maintenance. Each session corresponds to an independent thread. A greater value of this parameter may cause out-of-memory (OOM) errors.
spark.sql.retainedSessions
Other Settings
No
The other Spark settings. For more information, see Spark application configuration parameters and Spark Configuration.
None
Click Start.
Stop a Thrift server
If you want to use a job resource group that has a Thrift server started to run Spark SQL jobs in batch mode, you must stop the Thrift server first.
Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. On the Data Lakehouse Edition tab, find the cluster that you want to manage and click the cluster ID.
In the left-side navigation pane, choose Cluster Management > Resource Management. On the page that appears, click the Resource Groups tab.
On the Resource Groups tab, find the job resource group that you want to manage and click Configure Thrift Server in the Actions column.
In the Configure Thrift Server panel, click Stop.