When you create a Flink workspace, Flink creates directories in the associated OSS Bucket. These directories store data files that are required for or generated by Flink jobs, such as system checkpoints, job snapshots, logs, and JAR packages. This topic describes how to upload resources and the contents of each directory in the bucket.
OSS Bucket
Directory description
If you select an OSS Bucket as the storage type when you create a Flink workspace, Flink creates the following directories in the bucket to store the corresponding files.
Directory | Content |
artifacts | Stores uploaded JAR packages, additional dependencies, Python files, and Python dependencies. |
sql-artifacts | Stores files that are required by registered user-defined functions (UDFs) and connectors. |
flink-jobs | Stores High Availability (HA) information and system checkpoint information for Flink jobs. |
flink-savepoints | Stores snapshot files that are generated after you click Create Snapshot in the Flink development console. |
flink-sessionclusters | Stores HA information and system checkpoint information for session clusters. |
plan | Stores resource configuration information when you use expert mode. |
logs | Stores logs if you configure your job to output logs to OSS. |
File management
For workspaces that use an OSS Bucket, you can only upload, download, and delete files in the artifacts directory.
Log on to the Flink console.
Click Console in the Actions column of the target workspace.
In the Object Storage Service (OSS) console, configure a cross-origin resource sharing (CORS) rule for the OSS Bucket that is attached to the Flink workspace.
If a CORS rule is not configured, file uploads might fail because of cross-domain restrictions. When you configure the CORS rule, set Source to https://vvpsg.console.alibabacloud.com, select all methods for Allowed Methods, and set Allowed Headers to
*.
In the left navigation pane, click File Management. You can perform the following operations.
Operation
Description
Upload resource
Click Upload Resource and select a local file to upload. You can also drag the file to the resource list area.
After the upload is complete, the file path is in the format
oss://<Your attached OSS Bucket name>/artifacts/namespaces/<Project name>/<File name>. You can click the file name to copy the path.For more information about the project name, see View and delete a project.
Download resource
In the Actions column, click Download to download a file from the OSS Bucket to your local machine for viewing.
Delete resource
In the Actions column, click Delete to delete a file from the OSS Bucket.
File transfer acceleration
For workspaces that use an OSS Bucket, if you experience slow transfers of large files over long distances, you can enable the transfer acceleration feature. This feature is located in the upper-right corner of the page and improves upload and download speeds.
To use this feature, you must first enable the transfer acceleration feature for OSS. Enabling transfer acceleration incurs additional traffic fees. For more information about billing, see Transfer acceleration fees.
References
For more information about how to deploy SQL, JAR, and Python jobs, see Deploy a job.
For more information about how to view the detailed logs of a running job, see View startup and operational logs.
For more information about how to view the system checkpoints and snapshots of a job, see Manage job state sets.