Develop Flink jobs using VS Code - Realtime Compute for Apache Flink

The Realtime Compute for Apache Flink extension for VS Code streamlines the local development process for Flink jobs. You can use the extension to develop, deploy, and publish Flink jobs from your local environment. You can also sync job configurations from your online environment. This topic describes how to use the VS Code extension.

Prerequisites

You have created a workspace.
You have obtained an AccessKey pair with the permissions for the target namespace.
You have downloaded and installed VS Code v1.74.0+ .

Limitations

Log configuration is not supported.
Pulling existing jobs is not supported.

Install and configure the extension

Download and install the extension.
- Method 1: Go to the Visual Studio Marketplace and install Flink Realtime Compute Studio.
- Method 2:
  1. Open VS Code and click EXTENSIONS in the left navigation menu.
  2. In the search bar, enter Flink Realtime Compute Studio.
  3. Click Install.
In the menu bar, click File > Open Folder and select a local folder.

Configure the AccessKey information.

Workspace-wide

Click in the left menu bar. In the Configure AK&SK section, click Configure in a .vvprc file. The extension automatically creates a file with the .vvprc extension.

Enter the following configuration information.

{
  "accessKey": "xxx",
  "secretKey": "xxx",
  "namespace": "xxx",
  "workspace": "xxx",
  "regionId" : "xxx"
}

Parameter	Description	Notes
accessKey	The AccessKey ID of your account.	See How do I find my AccessKey ID and AccessKey secret?.
secretKey	The AccessKey secret of your account.	See How do I find my AccessKey ID and AccessKey secret?.
namespace	The name of the Flink namespace.	See View and delete a namespace.
workspace	The ID of the Flink workspace.	See How do I view information such as the workspace ID?.
regionId	The region ID.	See the region ID in Endpoints.

Globally

Click in the left menu bar.
Click Configure AK&SK > Configure in global user settings. You are redirected to the global settings of VS Code.

Select User or Workspace to configure the settings as needed.

Note

The User configuration applies to all projects and workspaces in the VS Code editor and is stored in the global configuration file. The Workspace configuration applies only to the currently open workspace.

Parameter	Description	Notes
Access Key	The AccessKey ID of your account.	See How do I view the AccessKey ID and AccessKey secret?
Secret Key	The AccessKey secret of your account.	See How do I view the AccessKey ID and AccessKey secret?
Namespace	The name of the Flink namespace.	See View and delete a namespace.
Workspace	The ID of the Flink workspace.	See How do I view information such as the workspace ID?
Region ID	The region ID.	See the region ID in Endpoints.

Note

Configuration priority: Current folder > Parent folder > Global.

After the configuration is applied, the name of the corresponding namespace appears in the status bar at the bottom of the job editor.

(Optional) Install the following extensions to improve your coding efficiency.
- Flink SQL: Install Flink SQL Language Features. This extension provides Flink SQL support.
- Flink configuration: Install YAML. This extension provides an auto-complete feature for YAML scripts.

Procedure

Step 1: Create a job

Method 1: Click in the left menu bar. In the QUICK START section, click Create a new deployment. Enter a name at the top and press Enter to create a .vvp file. This creates a job with default fields.
Method 2: In the file tree, create a .vvp file.

Step 2: Configure the job

Configure job settings visually:

Alternatively, click Raw File in the upper-left corner to configure the job in the source file. In the UI form, click a parameter name to navigate to the corresponding location in the source file.

Basic settings:

SQL job

Parameter	Description
Deployment Name	The name of the job.
Execution Mode	The deployment mode. Valid values: STREAM and BATCH.
Engine Version	The engine version to run the job. See Engine version. Use the recommended version.
Deployment Target	The destination resource queue for the job. For more information, see Manage resource queues.
Deployment Type	Select SQL.
SQL Script	Click Open in external editor to open the local editor and write your code. Note The editor opened from here is a virtual file, not a file in the actual file system. Because of this, if you reopen VS Code while the virtual file is still open, the editor window displays the `The editor could not be opened because the file was not found` error message. To fix this, click Open in external editor again.
Additional Dependencies	The URI of the additional dependencies required for the job.
Description	Enter remarks for the job.

JAR job

For more information about parameter settings, see Deploy a JAR job.

Parameter	Description
DeploymentName	The name of the job.
Execution Mode	The deployment mode. Valid values: STREAM and BATCH.
Engine Version	The engine version to run the job. See Engine version. Use the recommended version.
Deployment Target	The destination resource queue for the job. For more information, see Manage resource queues.
Deployment Type	The type of job to deploy. Select JAR.
JAR URI	The OSS path or URL of the JAR package. Make sure you have uploaded the dependency files to Artifacts in the Development Console or to a bound OSS bucket.
Entry Point Class	The entry point class of the program. If your JAR package does not specify a main class, enter the standard path of your entry point class.
Entry Point Main Arguments	Enter the input parameters to be invoked in the main method.
Additional Dependencies	The OSS path or URL of the destination additional dependency files. Only URLs that end with a file name are supported. Make sure you have uploaded the dependency files to Artifacts in the Development Console or to a bound OSS bucket.
Description	Enter remarks for the job.

Python job

For more information about parameter settings, see Deploy a Python job.

Parameter	Description
Deployment Name	The name of the job.
Execution Mode	The deployment mode. Valid values: STREAM and BATCH.
Engine Version	The engine version to run the job. See Engine version. Use the recommended version.
Deployment Target	The destination resource queue for the job. For more information, see Manage resource queues.
Deployment Type	The type of job to deploy. Select Python.
Python Uri	The address of the Python job file. The file can be a .py or .zip file.
Entry Module	The entry class for the program. Optional for .py files. Required for .zip files.
Entry Point Main Arguments	Enter the input arguments in the main method.
Python Libraries	Third-party Python packages, added to the PYTHONPATH of the Python worker process so that they can be directly accessed in Python UDFs. For more information, see Use third-party Python packages.
Python Archives	Archive files. For more information, see Use a custom Python virtual environment and Use data files.
Additional Dependencies	The OSS path or URL of the destination additional dependency files. Make sure you have uploaded the dependency files to Artifacts in the Development Console or to a bound OSS bucket.
Description	Enter remarks for the job.

Resources: See Configure job resources.
Parameters: See Parameters.

Step 3: Deploy the job

After you configure the job, click Create at the bottom of the page to deploy it. For an existing local job, click Update to deploy your changes.

For SQL jobs, the extension validates the SQL script, resource plan, and other job configurations, and then displays the validation results.

Step 4: Start the job

For a deployed job, the current job status is displayed in the upper-right corner. Click Start and select a startup mode. For more information, see Start a job.

Step 5: Perform job O&M

Open Online: View the detailed running status of the job or perform O&M operations in the Development Console.
Synchronize: Synchronize jobs from the console. Click Synchronize to open the job preview page (SQL draft updates not available). After confirmation, click Accept in the upper-right corner. The extension then overwrites the local job file with the job file from the cloud.
Cancel: Stop the job.

Realtime Compute for Apache Flink:Develop jobs using VS Code