All Products
Search

This Product

    Platform For AI:Create a DSW instance

    Document Center

    Platform For AI:Create a DSW instance

    Last Updated:Nov 19, 2024

    Data Science Workshop (DSW) is a cloud integrated development environment (IDE) that is provided by Platform for AI (PAI) for interactive programming. Before you use DSW, you must create a DSW instance. This topic describes how to create a DSW instance.

    Background information

    You can create and manage DSW instances in the PAI console. For more information, see the Create a DSW instance in the PAI console section in this topic.

    Prerequisites

    • The required permissions to use DSW are obtained. For more information, see Grant the permissions that are required to use DSW.

    • The general computing resources or Lingjun AI Computing Service (Lingjun) resources that you want to use to run DSW instances are prepared.

    • (Optional) An File Storage NAS (NAS) or Object Storage Service (OSS) dataset is created. For more information, see Create and manage datasets.

      Important

      After you create an OSS or NAS dataset for a DSW instance, you must obtain the permissions to access OSS or NAS. Otherwise, the DSW instance cannot read data from or write data to OSS or NAS. For more information, see Grant the permissions that are required to use DSW.

      • If you create a DSW instance by using the public resource group, DSW provides a disk storage quota. You can mount datasets or purchase additional disks to increase the storage size.

      • If you create a DSW instance by using a dedicated resource group, DSW provides non-persistent on-premises storage. You can mount datasets to persist data.

    Limits

    Each Alibaba Cloud account can purchase up to two GPUs per region. If the resource usage exceeds the limit, an error may occur. If you want to increase the quota, submit a ticket.

    Create a DSW instance in the PAI console

    1. Go to the Interactive Modeling (DSW) page.

      1. Log on to the PAI console.

      2. On the Overview page, select a region in the top navigation bar.

      3. In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the workspace that you want to manage.

      4. In the left-side navigation pane, choose Model Development and Training > Interactive Modeling (DSW).

    2. Click Create Instance.

    3. On the Create Instance page, configure the parameters that are described in the following table.

      4. Parameter

      Description

      Basic Information

      Instance Name

      The name of the DSW instance.

      Resource Information

      Resource Quota

      Valid values:

      • The default value of this parameter is Public Resource Group (Pay-as-you-go). You can select a CPU or GPU instance type. You can view and select the available instance types. For information about the metrics and descriptions of instance types, see Overview of instance families.

      • You can use a dedicated resource group of general computing resources to create DSW instances. If no dedicated resource group is available, click Associate Resource Quota to associate a dedicated resource group with the workspace.

      • You can use Lingjun resources to create instances. If no Lingjun resources are available, click Associate Resource Quota to associate Lingjun resources with the current workspace.

      Note

      If you use a dedicated resource group of general computing resources or a Lingjun resource group to create DSW instances, you must configure the following parameters:

      • Priority: Valid values: 1 to 9. A large value indicates a high priority.

      • CPU Affinity: If you enable the CPU affinity feature, processes in a container or pod can be bound to a specific CPU for execution. This prevents issues such as CPU cache misses and context switching, increases CPU utilization, and improves the application performance. You can enable the CPU affinity feature if your business has high requirements on performance and timeliness. The CPU affinity feature is available only in the China (Beijing) and China (Shenzhen) regions.

      Instance Type

      • If you select Public Resource Group (Pay-as-you-go) for Resource Quota, select the CPU, GPU, or free instance type based on your business requirements.

      • If you select a dedicated resource group for Resource Quota, configure the number of GPUs, number of vCPUs, memory, and priority based on your business requirements.

      Environment Information

      Image

      Valid values:

      • Alibaba Cloud Image: built-in official images. PAI provides built-in images for multiple versions of frameworks, including Python, TensorFlow, and PyTorch.

      • Custom Image: You can select an existing custom image. For information about how to create a custom image, see Custom images.

      • Image Address: You can use an image by entering a publicly accessible URL of the image. You can also enter the publicly accessible URL of the image in Create a Container Registry Personal Edition instance in the current region.

      System Disk

      • If you select Public Resource Group (Pay-as-you-go) for Resource Quota, set this parameter based on the following instructions:

        • System Disk: Each pay-as-you-go instance provides 100 GiB of free system disk for persistent storage. If the DSW instance is stopped for more than 15 days, the system disk will be cleared. If the free system disk does not meet your business requirements, you can expand the storage space of the disk. For information about the disk pricing, go to the PAI console.

          Warning

          • After the expansion, you cannot reduce the storage space. Proceed with caution.

          • After the expansion, the disk is not cleared if the instance is stopped and not recovered for more than 15 days. However, you continue to be charged for data storage.

          • If you delete the instance, the system disk is also released and the data stored in the disk is deleted. Make sure that you have backed up your data before you delete the instance.

        • Mount Settings: The storage space of the default system disk is limited. You can click Add and select a dataset that you want to mount to the instance to expand the storage space. You can mount OSS, File Storage NAS NAS, and Cloud Parallel File Storage (CPFS) datasets. If no datasets are available, click Create Dataset to create a dataset. For more information, see Create and manage datasets.

      • If you select a dedicated resource group for Resource Quota, set this parameter based on the following instructions:

        The system disk of the instance is used for temporary storage. The system disk is cleared when the instance is stopped or deleted. If you require persistent storage, click Add next to Dataset and select an available dataset, or click Add next to Mount Settings and select an OSS path.

        If no datasets are available, click Create Dataset to create a dataset. For more information about how to create a dataset, see Create and manage datasets.

      Note

      • You cannot mount multiple datasets to the same directory.

      • If you use a CPFS dataset, specify a virtual private cloud (VPC) for the DSW instance. The VPC must be the same as the VPC of the CPFS dataset. Otherwise, the instance may fail to be created.

      • If you set the Resource Quota parameter to a dedicated resource group, the first dataset that you mount to the instance must be a NAS dataset. The dataset is simultaneously mounted to the path that you specify and the default working directory /mnt/workspace/.

      Dataset

      You can click Add to mount a custom dataset. Custom OSS datasets support multiple mount modes and custom configurations. Public datasets support only the read-only mount mode.

      Mount Settings

      Click Add to mount an OSS path. OSS supports multiple mount modes and custom configurations.

      Working Directory

      The working directory is the startup directory of JupyterLab and WebIDE. The working directory is mounted to the /mnt/workspace directory.

      Network Information

      VPC Settings

      This parameter is available only if you set the Resource Quota parameter to Public Resource Group (Pay-as-you-go).

      For more information about how to configure networks for DSW instances, see DSW network configuration.

      To connect to a DSW instance over VPC, configure the VPC, vSwitch, and Security Group parameters. You can select a created VPC, or click Create VPC in the lower part of the drop-down list to create a VPC.

      Internet Access Gateway

      • Internet Access Gateway: You can select one of the following options:

        • Public Gateway: The DSW instance shares bandwidth with other instances in the cluster. The download rate is slow in high concurrency scenarios.

        • Private Gateway: The DSW instance uses dedicated bandwidth. You can configure the dedicated bandwidth based on your business requirements. If you select this option, you must create an Internet NAT gateway for the VPC that is associated with the DSW instance, associate an elastic IP address (EIP) with the DSW instance, and configure an SNAT entry. For more information, see Enable Internet access for a DSW instance by using a private Internet NAT gateway.

      • You need to configure the following parameters if you select a CPFS dataset for the Mount Settings parameter:

        • Enable All Options: By default, this option is not selected, which indicates that the system disables the VPCs that cannot connect to the CPFS dataset.

        • Hide Unavailable Options: If you select this option, the VPCs that cannot connect to the CPFS dataset are not displayed.

        Note

        If you use a CPFS dataset, you must specify a VPC for the DSW instance, and the VPC must be the same as the VPC of the CPFS dataset.

      Access Configuration

      Enable SSH

      You can configure this parameter if you use a VPC.

      After you enable SSH, you can directly log on to the DSW instance by using SSH based on the selected VPC. If you configure a custom image, you must make sure that sshd is installed on the custom image.

      SSH Public Key

      You can configure this parameter after you turn on Enable SSH.

      Note

      If you want to enable VPC and Internet logon, you must add the public keys of multiple clients. Separate public keys by pressing the Enter key. You can add up to 10 public keys.

      SSH Access Method

      You can configure this parameter after you turn on Enable SSH.

      • Access over VPC: the default access method. You can remotely connect to the DSW instance by using SSH from another terminal, such as an ECS instance in the VPC.

      • Access over Internet: You can enable Internet access and VPC access. After you select Access over Internet and configure the following parameters, you can connect to the DSW instance over SSH by using an on-premises CLI or another terminal.

        • NAT Gateway: Select the Internet NAT gateway that you created for the VPC.

        • EIP: Select the elastic IP address (EIP) that you created on the Internet NAT gateway.

      Custom Services

      Allow external access to custom services running in the instance. For more information, see Custom services access configuration.

      Advanced Information

      Visibility

      You can select Visible to the Instance Owner or Visible to the Current Workspace.

      Instance Owner

      Only the workspace administrator can change the instance owner.

      Instance RAM Role

      You can associate a RAM role with the DSW instance to enable the instance to access APIs of other cloud services based on temporary credentials of Security Token Service (STS). The temporary credentials are periodically updated. This ensures the security of your AccessKey pair and helps you implement fine-grained permission control and management by using RAM.

      Instance RAM roles include Default Roles of PAI, Custom Roles, and Does Not Associate Role.

      For more information about how to configure an instance RAM role, see Configure RAM roles for a DSW instance.

    4. After you confirm the configurations, click Yes.

    References

    • After you create a DSW instance, you can prepare the data files that are required for development. DSW supports multiple data source types, such as OSS, NAS, and MaxCompute. For more information, see Read and write data and file transfer. DSW also provides the data upload and download feature. For more information, see Upload or download data files.

    • For information about the features and workflow of DSW and how to get started with DSW, see DSW overview.

    • For information about the use cases of DSW, see DSW use cases.