Platform For AI:Upload or download data files

Background information

When you upload or download a data file in DSW, you can select a method based on the file size.

• If the file size is less than or equal to 5 GB, you can use one of the following methods to upload or download the data file:

  • Notebook

    Note

    DSW provides a file transfer station in Notebook. If you want to upload large models or large data files on your on-premises machine to a DSW instance, the file transfer station accelerates the upload process, and you are not charged additional fees. You can save and use a large file that you uploaded on multiple DSW instances that belong to the same RAM user.

  • WebIDE

• If the file size is greater than 5 GB, we recommend that you use ossutil to upload the file to an Object Storage Service (OSS) bucket. Then, create an OSS dataset based on the data file and mount the dataset to a DSW instance. This way, the DSW instance can read data from the data file. For more information, see Read and write dataset data.

Limits

Take note of the following limits when you use the file transfer station of Notebook to upload data files:

• You can upload up to five data files at a time. If you upload more than five data files, the excess files wait in a queue to be uploaded.

• You cannot upload a folder. If you want to upload a folder, we recommend that you compress the folder into a package and upload the package.

• The default validity period of data files in the file transfer station is seven days. Before a data file expires, you can extend the validity period of the file for another seven days with one click.

  You cannot extend the validity period of a data file that expires.

• The capacity of the file transfer station is 10 GB. You can store up to 1,000 data files in the file transfer station.

Prerequisites

A DSW instance is created. For more information, see Create a DSW instance.

Upload or download data files on the Notebook page

1. Go to the DSW development environment.

   1. Log on to the Platform for AI (PAI) console.

   2. In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the workspace that you want to manage.

   3. In the upper-left corner of the page, select the region in which you want to use DSW.

   4. In the left-side navigation pane, choose Model Training > Data Science Workshop (DSW). On the page that appears, click Enter Data Science Workshop (DSW).

   5. Optional: On the Data Science Workshop (DSW) page, enter the name of a DSW instance or a keyword in the search box to search for the instance.

   6. Click Open in the Actions column.

2. Upload or download a data file on the Notebook page.

   Area	Upload/Download	Operation
   Area 1	Upload	Click the icon or drag the data file to the blank area of the file list to upload the file. The system determines the upload method based on the file size. If the file size is less than or equal to 10 MB, the data file is uploaded to the DSW instance by using the current browser. If the file size is greater than 10 MB but less than or equal to 5 GB, the data file is automatically uploaded to the file transfer station and saved to the instance.
   Area 2	Download	Right-click the data file and select Download. The data file is downloaded to your on-premises machine.

   

3. (Optional) Click the icon in the left-side toolbar to go to the File Transfer Station page. On the page, you can view the transfer list or perform operations on data files.

   Area	Operation
   Area 1	Click the icon to view the transfer list in the file transfer station.
   Area 2	Click the icon to the right of the data file that you want to manage to perform the following operations: After the file transfer is complete, the data file is automatically saved to the instance. If you want to repeatedly save the data file to the current instance, click Save to Instance. If you want to use the data file on other instances that belong to the same RAM user, click Save to Instance. The default validity period of data files in the file transfer station is seven days. You can click Extend Expiration to extend the validity period of a data file before the validity period ends. Click Delete to delete the data file from the file transfer station. The data file is not deleted from the file list.

   

Upload or download data files on the WebIDE page

1. Open the DSW instance that you want to manage and click WebIDE in the top navigation bar of the page that appears.

2. On the WebIDE page, click the icon in the left-side toolbar to upload data files to or download data files from the file list.

   • Upload a data file: Right-click the directory in which you want to save the data file, select Upload from the shortcut menu, and then upload the file as prompted.

   • Download a data file: Right-click the data file that you want to download and select Download from the shortcut menu.

     Note

     You can download only data files. You cannot download folders. If you want to download a folder, compress the folder on the Terminal page, right-click the package on the WebIDE page, and then select Download from the shortcut menu.

References

• DSW is a cloud-based integrated development environment (IDE) for machine learning in PAI. After you upload a data file, you can start to use DSW. For more information, see DSW overview.

• DSW supports a variety of data sources, such as OSS, File Storage NAS (NAS), and MaxCompute data sources. For more information, see Read and write data.