Data Science Workshop (DSW) is a cloud integrated development environment (IDE) that is provided by Platform for AI (PAI) for interactive programming of machine learning. Before you use DSW, you must create a DSW instance. This topic describes how to create a DSW instance.
Background information
If you use DSW to develop models, you can create and manage DSW instances in the PAI console.
Prerequisites
The required permissions are granted.
Activate PAI and create a workspace by using an Alibaba Cloud account. Log on to the PAI console, select a region where you want to activate PAI in the upper-left corner of the page, and then complete authentication, authorization, and service activation. For more information, see Activate PAI and create a default workspace.
Authorize the operation account. If you use an Alibaba Cloud account to manage DSW instances, skip this step. If you use a RAM account, you must authorize the operation account.
(Optional) A dedicated resource group is prepared.
After you complete configurations in Step 1, a public resource group is prepared. If you want to use a dedicated resource group, you must purchase dedicated resources and allocate resource quotas.
If you want to use a dedicated resource group of general computing resources, see Create a dedicated resource group and purchase general computing resources and General computing resource quotas.
If you want to use a dedicated resource group of Lingjun resources, see Create a resource group and purchase Lingjun resources and Lingjun resource quotas.
(Optional) A dataset is prepared.
The default data storage size of public and dedicated resource groups is limited and does not support persistent storage. If you want to expand the storage space of an instance or persistently store data, you can mount a File Storage NAS (NAS) or Object Storage Service (OSS) dataset or an OSS path. For more information about how to create a dataset, see Create and manage datasets.
ImportantIf you create a DSW instance by using the public resource group, data is stored in free disks with limited storage space. Data is cleared after the instance is deleted or stopped for more than 15 days.
If you create a DSW instance by using the dedicated resource group, data is stored in the system disk of the instance. Data in the temporary storage is cleared after the instance is deleted or stopped.
(Optional) A custom image is used.
DSW provides various types of preset official images, such as PyTorch, TensorFlow, and ModelScope. If you want to use a custom image to meet specific development requirements, see Custom images.
Limits
Each Alibaba Cloud account can purchase up to two GPUs per region. If the resource usage exceeds the limit, an error may occur. If you want to increase the quota, submit a ticket.
Create an instance
After you create an instance by using public resources, you are charged based on the subscription duration of the instance. The billing stops only after you stop or delete the instance. For more information about billing rules, see Billing of DSW.
Create a DSW instance in the PAI console
Go to the Interactive Modeling (DSW) page.
Log on to the PAI console.
On the Overview page, select a region in the top navigation bar.
In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the workspace that you want to manage.
In the left-side navigation pane, choose .
Click Create Instance.
On the Create Instance page, configure the following key parameters.
Parameter
Description
Basic Information
Instance Name
The name of the DSW instance.
Tag
The instance tag. You can tag the instance based on your business requirements. This facilitates multi-dimensional resource searching, resource locating, batch operation, and cost allocation.
Resource Information
Resource Type
Valid values:
Public Resources: Only the pay-as-you-go billing method is supported. You cannot change the billing method from pay-as-you-go to subscription.
Resource Quota: You can select general computing resources or intelligent computing Lingjun resources. If resources you want to use are available, click Associate Resource Quota to configure the resources.
The parameters that you can configure vary based on the resource types.
If Resource Type is set to Public Resources, you must configure the following parameters:
Instance Type: You can select the CPU, GPU, or free instance. For more information, see Overview of instance families.
(Optional) Bidding Purchase: If you create a DSW instance by using the public resources, you can use preemptible instances. This reduces costs.
This parameter is available only in the China (Hangzhou), China (Shanghai), China (Beijing), China (Ulanqab), China (Shenzhen), China (Guangzhou), Japan (Tokyo), and Singapore regions.
If Resource Type is set to Resource Quota, you must configure the following parameters:
Resource Quota: You can select general computing resources or intelligent computing Lingjun resources.
Instance Type: You can select the CPU, GPU, or free instance based on your business requirements.
Priority: Valid values: 1 to 9. A large value indicates a high priority.
CPU Affinity: If you enable CPU affinity, processes in containers or pods can be bound to specific CPU cores. This reduces CPU cache misses and context switches and increases CPU utilization and performance in scenarios that require high performance and low latency.
This parameter is available only in the China (Beijing) and China (Shenzhen) regions.
Environment Information
Image
Valid values:
Alibaba Cloud Image: the preset official image. Python, TensorFlow, and PyTorch versions of images are supported.
Custom Image: a custom image that you created. For more information about how to add a custom image, see Custom images.
Image Address: You can use an image by entering a publicly accessible URL of the image. You can also enter the publicly accessible URL of the image in Container Registry Personal Edition in the current region. For more information, see Create a Container Registry Personal Edition instance.
System Disk
If you set Resource Type to Public Resources or select subscription general computing resources that provide more than 2 CPU cores and more than 4 GB of memory or GPUs) for Resource Quota:
Each instance is provided with 100 GiB of free system disk for persistent storage. If the instance is stopped for more than 15 days, the data in the disk will be cleared. The disk storage space can be expanded. For information about the disk storage expansion pricing, go to the PAI console.
WarningAfter the expansion, you cannot reduce the storage space. Proceed with caution.
After the expansion, the disk is not cleared if the instance is stopped and not recovered for more than 15 days. However, you continue to be charged for data storage.
If you delete the instance, the system disk is also released and the data stored in the disk is deleted. Make sure that you have backed up your data before you delete the instance.
If you want to permanently store the data, you can configure Dataset Mounting or enter an OSS path in the Mount Path field of Storage Path Mounting.
Dataset Mounting
Click Custom Dataset to mount a custom dataset that has been created. OSS custom datasets support multiple mount modes and custom configurations. Public datasets support only the read-only mount mode.
Mount Path is the location where the dataset is mounted in the DSW instance. For example, the default mount path
/mnt/data/indicates that the dataset is mounted in the/mnt/data/directory within the DSW instance.NoteYou cannot mount multiple datasets to the same path.
If you use a Cloud Parallel File Storage (CPFS) dataset, specify a virtual private cloud (VPC) for the instance. The VPC must be the same as the VPC of the CPFS dataset. Otherwise, the DSW instance may fail to be created.
If you set the Resource Quota parameter to a dedicated resource group, the first dataset that you mount to the instance must be a NAS dataset. The dataset is simultaneously mounted to the path that you specify and the default working directory/mnt/workspace/.
Storage Path Mounting
Click OSS to mount an OSS path to DSW. OSS supports multiple mount modes and custom configurations.
Mount Path is the same as the Mount Path of Dataset Mounting.
Working Directory
The working directory is the startup directory of Notebook and WebIDE. The working directory is mounted to the
/mnt/workspacedirectory.Network Information
VPC
This parameter is available only if you set the Resource Type parameter to Public Resources.
To connect to a DSW instance over VPC, you must configure this parameter together with the vSwitch and Security Group parameters. For more information about configurations in different scenarios, see DSW network configuration.
Internet Gateway
You can select one of the following options for Internet Gateway:
Public Gateway: The public bandwidth is shared by multiple DSW instances in the cluster. The download rate is slow in high concurrent scenarios.
Private Gateway (recommended): The DSW instance uses dedicated bandwidth. You can configure the dedicated bandwidth based on your business requirements. If you select this option, you must create an Internet NAT gateway for the VPC that is associated with the DSW instance, associate an elastic IP address (EIP) with the DSW instance, and configure an SNAT entry. For more information, see Enable Internet access for a DSW instance by using a private Internet NAT gateway.
You need to configure the following parameters if you select a CPFS dataset for the Custom Dataset parameter:
Enable All Options: By default, this option is not selected, which indicates that the system disables the VPCs that cannot connect to the CPFS dataset.
NoteIf you use a CPFS dataset, you must specify a VPC for the DSW instance, and the VPC must be the same as the VPC of the CPFS dataset.
Access Configuration
Enable SSH
You can configure this parameter if you use a VPC.
After you enable SSH, you can directly log on to the DSW instance by using SSH based on the selected VPC. If you configure a custom image, you must make sure that sshd is installed on the custom image.
SSH Public Key
You can configure this parameter after you turn on Enable SSH.
NoteIf you want to enable VPC and Internet logon, you must add the public keys of multiple clients. Separate public keys by pressing the Enter key. You can add up to 10 public keys.
SSH Access Method
You can configure this parameter after you turn on Enable SSH.
Access over VPC: the default access method. You can remotely connect to the DSW instance by using SSH from another terminal, such as an ECS instance in the VPC.
Access over Internet: After you select Access over Internet, the Internet access method is added. You can configure the following parameters and connect to the DSW instance over SSH by using an on-premises CLI or another terminal.
NAT Gateway: Select the Internet NAT gateway that you created for the VPC.
EIP: Select the EIP that you created on the Internet NAT gateway.
Custom Services
Allow external access to custom services running in the instance. For more information, see Custom services access configuration.
Create Private Zone in VPC
Create a private domain (Private Zone). You can use the Private Zone in this VPC to access the SSH service or other custom services of the current instance. This avoids the inconvenience caused by the changing IP address of the instance. Note that the Private Zone will incur fees. For more information, see Billing.
Advanced Information
Visibility
You can select Visible to the Instance Owner or Visible to the Current Workspace.
Instance Owner
Only the workspace administrator can change the instance owner.
Instance RAM Role
When you access other cloud resources from a DSW instance, you can associate a RAM role with the instance. This method allows you to use temporary Security Token Service (STS) tokens instead of AccessKey pairs to access the resources, which effectively reduces the risk of AccessKey pair leaks.
You can select one of the following options for this parameter:
Default Roles of PAI: The default roles of PAI have the permissions to access PAI services, MaxCompute, and OSS. If you use the temporary credentials issued by the default roles of PAI, you are granted the same permissions as the DSW instance owner when you access PAI services and MaxCompute tables. When you access OSS, you can access only the bucket that is configured as the default storage path for the current workspace.
Custom Roles: If you want to perform customized or fine-grained permission control, you can configure custom roles.
Does Not Associate Role: If you want to directly access resources of other cloud services by using an AccessKey pair, you can choose not to associate a role with the instance.
For more information about how to configure an instance RAM role, see Configure RAM roles for a DSW instance.
After you confirm the configurations, click Yes.
References
After you create an instance, you can prepare the data files that are required for development. DSW supports multiple data sources, including OSS, NAS, and MaxCompute. For more information, see Read data from and write data to OSS and MaxCompute.
You can also use a DSW instance to upload and download small-sized data files. For more information, see Upload or download data files.
For information about the features and workflow of DSW and how to get started with DSW, see DSW overview.
For information about the use cases of DSW, see DSW use cases.
After creating an instance, you can save the environment of a current running DSW instance as a custom image for future use. For more information, see Create a DSW instance image.