The personal development environment in Data Studio is an account-level cloud development instance that integrates NAS storage, Git code management, and Python/Notebook ecosystems. It supports local script execution, online debugging, and task submission. With flexible custom images and external service extensibility, it provides efficient and customizable end-to-end support for data processing, model training, and collaborative development, helping to improve code quality and development efficiency. This topic describes how to use the personal development environment.
Background information
Data Studio allows you to create a personal development environment instance at the account level. After network connections are established, you can access a Git repository and NAS with ease to clone code in the Git repository, and develop and debug Python code and notebook code online. You can also submit code to the project workspace for scheduling.
Personal development environment instances provide the following features:
Support SQL, AI notebook, and Python ecosystems, allow you to run Python and Shell scripts on your on-premises machine, and provide a code debugging feature.
Integrate Git code management, allowing you to easily clone, push, and manage code.
Support the integration with NAS for data management and access.
Allow you to use custom images and support the connection with various external services to improve flexibility and extensibility.
Data Studio provides an efficient, flexible, and powerful development environment that allows you to perform operations, such as data processing, analysis, and model training, in a more convenient manner to improve the development efficiency and code quality.
Billing
When creating a personal development environment instance, you must select a resource group and set a compute unit (CU) quota. Charges for the resource group are based on the number of CUs and the duration of operation. For more information, see Serverless resource group billing.
When your personal development environment instance is in the Running state and utilizes a pay-as-you-go resource group, computing fees are calculated based on
resource quota × instance running time. Conversely, if the instance is part of a subscription resource group, it consumes the resource group's available quota.When the personal development environment instance is not needed, please stop the personal development environment instance in the management instance promptly.
Prerequisites
You have created a workspace and selected the Join Data Development (Data Studio) (New Version) Public Preview.
You have created a resource group and bound it to the workspace. For more information, see Add and use Serverless resource groups.
Considerations
You can select and use only a personal development environment instance that is created by using the current Alibaba Cloud account. You cannot select or use a personal development environment instance that is created by using another Alibaba Cloud account.
Each member in a workspace can create up to
10personal development environment instances.Descriptions for file deletion from the storage space of a personal development environment instance:
Files mounted in the NAS space through the personal development environment instance (default location at
/mnt/data) do not go to the DataStudio recycle bin when deleted. However, if the file storage console has the recycle bin feature activated, deleted files will be moved to the file system's recycle bin.Files deleted from the built-in storage space of the personal development environment instance (default location:
/mnt/workspace) do not enter the recycle bin of Data Studio.
Each personal development environment instance provides 30 GiB of free disk (the built-in storage space of the personal development environment instance). The disk will be reclaimed 15 days after the instance is stopped. Please save your personal code files in time.
Feature entry
Go to the Workspaces page in the DataWorks console. In the top navigation bar, select a desired region. Find the desired workspace and choose in the Actions column.
Click the top Personal Development Environment · Please Select and choose an existing personal development environment instance or create a new one as required.
Create a personal development environment instance
If you don't have a personal development environment instance, you can create one by clicking Go To Create.
If you already have a personal development environment instance, click Manage Instance, and then click Create Instance in the list of personal development environment instances.
Parameters that are involved in creating a personal development environment instance:
Required parameters
Parameter | Description |
Instance Name | The name of the personal development environment instance. You can specify a custom name. |
Resource Group | Select a serverless resource group for the personal development environment instance. |
Resource Type | Support configuring resources of CPU type and GPU type. |
Resource Quota | Select the resource specifications for the personal development environment instance. Select based on your business requirements. After selecting the specifications, the following will be displayed:
You can assign the Resource Limit or CU Guarantee that the current personal development environment instance can use in the Quota Management of the serverless resource group. Important Please follow the Resource Quota information here. When the personal development environment instance is running, if the instance uses a pay-as-you-go resource group, the computing fee is charged based on |
Select Image | Select an image based on your business requirements.
|
System Disk | Each personal development environment instance provides 30 GiB of free disk (the built-in storage space of the personal development environment instance). The disk will be reclaimed 15 days after the instance is stopped. Please save your personal code files in time. DataWorks provides 30 GiB of free disk for each personal development environment instance. The disk will be reclaimed 15 days after the instance is stopped. Please save your personal code files in time. The default path for the built-in storage space of the personal development environment instance is |
Optional parameters
Add Storage Source
You can Add Storage Source to mount a NAS distributed file system instance within your personal development environment, enabling access to files stored on NAS for development in Data Studio. Additionally, scripts and files created in Data Studio can be persistently stored on the mounted NAS instance.
Parameter
Description
Data Storage
Support file storage (General-purpose NAS) and file storage (Extreme NAS).
File System
Select an existing file system.
File System Mount Target
Select a mount target in the file system. If none exists, click Create Mount Target.
File System Path
Enter the mount path of the file system storage space and the subpath under the file system. You can go to the NAS file system console to query the specific file system path. For the procedure, see: Query ECS Mount Status.
Enter the Mount Path from position ① in the figure into the Default Mount Path parameter.
Enter the NAS Directory from position ② in the figure into the File System Path parameter. If the file system path does not exist in the file system, the personal development environment instance cannot be created.
Default Mount Path
Network Settings
Network settings are optional. You can configure network settings in the following business scenarios:
To access resources within a specific VPC via the intranet, configure the appropriate Virtual Private Cloud, Security Group, and Vswitch here.
If you do not need to access resources in a VPC, you do not need to specify a VPC. By default, a personal development environment supports Internet access.
To access resources within a VPC along with those on the Internet, configure the VPC here and set up a Dedicated Internet Gateway for the VPC.
Advanced Information
Parameter
Description
Instance RAM Role
You can associate a RAM role with a personal development environment instance to enable the instance to access other cloud services based on temporary credentials of Security Token Service (STS). The temporary credentials are periodically updated. This ensures the security of your AccessKey pair and helps you implement fine-grained permission management by using RAM. The default selection is Dataworks Default Role.
For more information about roles, see Configure Instance RAM Role for Personal Development Environment.
Stop a personal development environment instance
When your personal development environment instance is in the Running state, computing fees for instances using a pay-as-you-go resource group are calculated based on
resource quota × instance running time. Instances using a subscription resource group consume the available quota from the resource group.When the personal development environment instance is not needed, please stop the personal development environment instance in the management instance promptly.
Each personal development environment instance provides 30 GiB of free disk (the built-in storage space of the personal development environment instance). The disk will be reclaimed 15 days after the instance is stopped. Please save your personal code files in time.
The personal development environment supports multiple ways to stop an instance:
Method 1: set a space-level automatic shutdown policy
The Alibaba Cloud account or space administrator can go to the management center to set an automatic shutdown policy for all personal development environment instances in the space. When the personal development environment instances in the space meet the space-level automatic shutdown policy, they will shut down automatically.
Go to the SettingCenter page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Management Center.
On the workspace details page, locate the Personal Development Environment Instance section and set the Automatic Shutdown Policy according to your needs.
Parameter configuration description:
Idle Duration is a mandatory parameter measured in hours.
You must configure at least one of the following: GPU Utilization or CPU Utilization.
NoteThe shutdown policy defined here applies to all running personal development environment instances in the current space. The policy takes effect immediately after being saved.
Modifying the automatic shutdown configuration of a personal development environment instance will recalculate the idle duration of the personal development environment instance.
Method 2: set a scheduled shutdown
In the Data Studio interface, click the top Personal Development Environment · Please Select and select Manage Instance.
In the instance list, locate the instance you want to stop automatically and click the Operation column's Scheduled Shutdown. Configure the Automatic Shutdown Time and then click Confirm.
Method 3: manually stop an instance
In the Data Studio interface, click the Personal Development Environment · Please Select at the top and select Manage Instance.
In the instance list, locate the instance you want to stop and click on the Operation column's Stop button.
Delete a personal development environment instance
To release a personal development environment instance you've created, locate the instance in the list, click the Operation column's Delete button, and confirm its deletion in the pop-up dialog box.
View resource utilization of a personal development environment instance
Hover the mouse over the resource utilization summary information area after the personal development environment instance name to view detailed information on each metric.
More references
You can create a custom image from a personal development environment instance and then use this image for development on other nodes to meet your business objectives. For more information on how to create and utilize images from personal development environment instances, see Create an Image from a Personal Development Environment.
For instructions on how to connect a personal development environment to a Git repository, clone, push, and manage code, see Connect a Personal Development Environment to a Git Repository.