E-MapReduce:Manage SQL computes

Create an SQL compute

After you create an SQL compute, you can select this compute when you create an SQL job.

1. Go to the Compute page.

   1. Log on to the EMR console.

   2. In the left-side navigation pane, choose EMR Serverless > Spark.

   3. On the Spark page, click the name of the workspace that you want to manage.

   4. In the left-side navigation pane of the EMR Serverless Spark page, click Compute.

2. On the Compute page, click Create SQL Compute.

3. On the Create SQL Compute page, configure the parameters described in the following table and click Create.

   Important

   It is recommended to set the concurrency limit of the selected deployment queue to be at least the size of the resources required by the Notebook Compute. Please refer to the values displayed in the console for specifics.

   Parameter	Description
   Name	The SQL compute name. The name must be 1 to 64 characters in length and can contain letters, digits, hyphens (-), underscores (_), and spaces.
   Resource Queue	The resource queue in which the SQL compute is deployed. Select a resource queue from the drop-down list based on your business requirements. Only resource queues that are available in the development environment or resource queues that are available in both the development and production environments are displayed in the drop-down list. For more information about resource queues, see Manage resource queues.
   Engine Version	The version of the engine that is used by the SQL compute. For more information about engine versions, see Engine versions.
   Auto Stop	By default, this feature is enabled. By default, the system automatically releases the SQL compute if the SQL compute is not used to run any jobs in the last 45 minutes.
   spark.driver.cores	The number of CPU cores that are used by the driver of the Spark application. Default value: 1 CPU.
   spark.driver.memory	The size of memory that is available to the driver of the Spark application. Default value: 3.5 GB.
   spark.executor.cores	The number of CPU cores that can be used by each executor. Default value: 1 CPU.
   spark.executor.memory	The size of memory that is available to each executor. Default value: 3.5 GB.
   spark.executor.instances	The number of executors that are allocated to the Spark application. Default value: 2.
   Dynamic Allocation	By default, this feature is disabled. After you enable this feature, you must configure the following parameters: Min executors: Default value: 2. Max executors: If you do not configure spark.executor.instances, this parameter is set to the default value 10.
   More memory configurations(Click to expand)	spark.driver.memoryOverhead: the size of non-heap memory that is available to each driver. Default value: 1 GB. spark.executor.memoryOverhead: the size of non-heap memory that is available to each executor. Default value: 1 GB. spark.memory.offHeap.size: the size of non-heap memory that is available to the Spark application. Default value: 1 GB. This parameter is valid only if spark.memory.offHeap.enabled is set to true. By default, this parameter is valid and set to 1 GB only if the Fusion engine is used.
   Spark Configurations	The Spark configurations. Separate the configurations with spaces, such as spark.sql.catalog.paimon.metastore dlf.

4. Click Start in the Actions column of the SQL compute that you created.

References

• For information about operations related to resource queues, see Manage resource queues.

• For information about the roles and permissions supported by SQL computes, see Manage users and roles.

• For information about how to develop an SQL job, see Get started with SQL jobs.