To improve the efficiency and flexibility of Apache Paimon data management, such as small file merging and data sorting, in the development console of Realtime Compute for Apache Flink, you must deploy relevant packages in advance. This topic describes how to deploy relevant packages in the development console of Realtime Compute for Apache Flink to manage Apache Paimon data in an efficient manner.
Limits
Only Realtime Compute for Apache Flink that uses Ververica Runtime (VVR) 8.0.5 or later supports Apache Paimon tables.
Procedure
Click the following links to download the JAR packages that are required for the management of Apache Paimon data.
In the left-side navigation pane of the development console of Realtime Compute for Apache Flink, click Artifacts. On the Artifacts page, click Upload Artifact to upload the two JAR packages that you downloaded in Step 1.
In the left-side navigation pane, click . On the Deployments page, click . In the Create Deployment dialog box, configure the parameters.
The following table describes the parameters.
Parameter
Description
Example
Deployment Mode
Select Stream Mode or Batch Mode based on your business requirements.
Batch Mode
Deployment Name
Enter the name of the JAR deployment that you want to create.
test
Engine Version
Select a version of Realtime Compute for Apache Flink that uses VVR 8.0.5 or later.
vvr-8.0.5-flink-1.17
JAR URI
Upload the paimon-flink-action JAR package.
Select the paimon-flink-action JAR package that is uploaded in Step 2.
Entry Point Class
The entry point class of the program.
Leave this parameter empty.
Entry Point Main Arguments
An input parameter. You can call the parameter in the main method.
Specify the parameter based on your business requirements. For more information, see the documentation of related features. For example, you can specify the parameter to sort data. For more information, see Optimize deployment consumption.
Additional Dependencies
Specify the path or file name of the additional dependency file.
Select the paimon-ali-vvr package that is uploaded in Step 2.
Click Deploy.
On the page, find the deployment that you create, and click Start in the Actions column. For more information about how to start a deployment, see Start a deployment.
References
After a Realtime Compute for Apache Flink deployment is started, you must configure the parameters to run the deployment. For more information, see Configure a deployment.
Some parameters for deployment running can be dynamically updated to reduce the service interruption time caused by the startup and cancellation of a deployment. For more information, see Dynamically update the parameter configuration for dynamic scaling.
You can take different measures to optimize the performance of Apache Paimon primary key tables and Append Scalable tables. For more information about common optimization methods, see Optimize the performance of Apache Paimon tables.