E-MapReduce:Manage notebook sessions

Create a notebook session

After you create a notebook session, you can select the session when you develop a notebook.

1. Go to the Notebook Sessions tab.

   1. Log on to the EMR console.

   2. In the left-side navigation pane, choose EMR Serverless > Spark.

   3. On the Spark page, find the desired workspace and click the name of the workspace.

   4. In the left-side navigation pane of the EMR Serverless Spark page, choose Operation Center > Sessions.

   5. Click the Notebook Sessions tab.

2. On the Notebook Sessions tab, click Create Notebook Session.

3. On the Create Notebook Session page, configure parameters and click Create. The following table describes the parameters.

   Important

   We recommend that you set the Maximum Concurrency parameter of the resource queue that you use to a value that is greater than or equal to the number of compute units (CUs) required by the notebook session. You can view the value of the Maximum Concurrency parameter in the EMR console.

   Parameter	Description
   Name	The name of the notebook session. The name must be 1 to 64 characters in length and can contain letters, digits, hyphens (-), underscores (_), and spaces.
   Resource Queue	The resource queue that is used to deploy the notebook session. Select a resource queue from the drop-down list. Only resource queues that are available in the development environment and resource queues that are available in both the development and production environments are displayed in the drop-down list. For more information about resource queues, see Manage resource queues.
   Engine Version	The version of the engine that is used by the notebook session. For more information about engine versions, see Engine versions.
   Use Fusion Acceleration	Specifies whether to enable Fusion acceleration. The Fusion engine helps accelerate the processing of Spark workloads and lower the overall cost of tasks. For more information about billing, see Billing. For more information about the Fusion engine, see Fusion engine.
   Runtime Environment	You can select a custom environment that is created on the Runtime Environments page. When the notebook session is started, the related libraries are pre-installed based on the environment that you select. Note You can select only a ready runtime environment.
   Automatic Stop	By default, this switch is turned on. You can configure the time at which you want the notebook session to automatically stop after the notebook session becomes inactive.
   spark.driver.cores	The number of vCPU cores that are used by the driver of the Spark application. Default value: 1 vCPU core.
   spark.driver.memory	The size of memory that is available to the driver of the Spark application. Default value: 3.5 GB.
   spark.executor.cores	The number of vCPU cores that can be used by each executor. Default value: 1 vCPU core.
   spark.executor.memory	The size of memory that is available to each executor. Default value: 3.5 GB.
   spark.executor.instances	The number of executors that are allocated to the Spark application. Default value: 2.
   Dynamic Resource Allocation	By default, this feature is disabled. After you enable this feature, you must configure the following parameters: Minimum Number of Executors: Default value: 2. Maximum Number of Executors: If you do not configure the spark.executor.instances parameter, the default value 10 is used.
   More Memory Configurations	spark.driver.memoryOverhead: the size of non-heap memory that is available to each driver. If you leave this parameter empty, Spark automatically assigns a value to this parameter based on the following formula: max(384 MB, 10% × spark.driver.memory). spark.executor.memoryOverhead: the size of non-heap memory that is available to each executor. If you leave this parameter empty, Spark automatically assigns a value to this parameter based on the following formula: max(384 MB, 10% × spark.executor.memory). spark.memory.offHeap.size: the size of off-heap memory that is available to the Spark application. Default value: 1 GB. This parameter is valid only if you set the spark.memory.offHeap.enabled parameter to true. By default, if you use the Fusion engine, the spark.memory.offHeap.enabled parameter is set to true and the spark.memory.offHeap.size parameter is set to 1 GB.
   Spark Configuration	The configurations of Spark. Separate the configurations with spaces. Example: spark.sql.catalog.paimon.metastore dlf.

4. Click Start in the Actions column of the notebook session that you created.

References

• For information about the operations related to resource queues, see Manage resource queues.

• For information about the roles and permissions supported by sessions, see Manage users and roles.

• For information about how to develop a notebook, see Get started with notebook development.