All Products
Search
Document Center

E-MapReduce:Manage runtime environments

Last Updated:Oct 30, 2024

By default, Matplotlib, NumPy, and Pandas are installed in a notebook session. If you want to use other third-party libraries, you can create a runtime environment.

Limits

You can use a runtime environment only in notebook sessions.

Create a runtime environment

  1. Go to the Runtime Environments page.

    1. Log on to the E-MapReduce (EMR) console.

    2. In the left-side navigation pane, choose EMR Serverless > Spark.

    3. On the Spark page, find the desired workspace and click the name of the workspace.

    4. In the left-side navigation pane of the EMR Serverless Spark page, click Runtime Environments.

  2. Click Create Runtime Environment.

  3. On the Create Runtime Environment page, configure the parameters. The following table describes the parameters.

    Parameter

    Description

    Name

    The name of the runtime environment.

    Description

    The description of the runtime environment.

    Queue for Environment Initialization

    The queue used to initialize the runtime environment. One CPU core and 4 GB of memory are used to initialize the runtime environment after you create the runtime environment. After the runtime environment is initialized, the resources are automatically released.

  4. Add a library.

    1. Click Add Library in the Libraries section.

    2. In the Create Library dialog box, configure the Source Type parameter and other parameters and then click OK.

      Parameter

      Description

      PyPI

      If you set the Source Type parameter to PyPI, you must configure the PyPI Package parameter. In the PyPI Package field, enter the name and version of the library. If you do not specify the library version, the latest version is installed. By default, the source address that is provided by Alibaba Cloud to download the package is used.

      Example: Plotly or Plotly==4.9.0.

      Workspace Resource

      If you set the Source Type parameter to Workspace Resource, you must configure the Workspace Resources parameter. Select a file resource from the Workspace Resources drop-down list. If no resource is available, upload a file resource on the Files page.

      The following file types are supported: .zip, .tar, .whl, .tar.gz, and .jar.

      OSS Resource

      If you set the Source Type parameter to OSS Resource, you must configure the OSS Resources parameter. In the OSS Resources field, enter the path of an object in an Object Storage Service (OSS) bucket.

      The following file types are supported: .zip, .tar, .whl, .tar.gz, and .jar.

  5. Click Create.

    After you create the runtime environment, the runtime environment is initialized.

Modify a runtime environment

You can modify a runtime environment to update the libraries that you added in the runtime environment.

  1. On the Runtime Environments page, find the runtime environment that you want to modify and click Edit in the Actions column.

  2. On the Modify Runtime Environment page, modify the configuration items of the runtime environment.

  3. Click Save Changes.

    After you save the changes, the runtime environment is reinitialized based on the new configurations.

    Note

    After a runtime environment is reinitialized, the changes do not immediately take effect in notebook sessions that use the runtime environment. To apply the changes to a notebook session, restart the notebook session.

Use the runtime environment

After the runtime environment enters the Ready state, you can use the runtime environment in notebook sessions. For more information, see Manage notebook sessions.