All Products
Search
Document Center

Realtime Compute for Apache Flink:Code templates

Last Updated:Sep 12, 2024

Fully managed Flink provides various code templates. Each code template provides use scenarios, code samples, and instructions for you. Code templates can help you quickly understand the features and related syntax of fully managed Flink and implement your own business logic. This topic describes the usage scenarios for code templates and how to use a code template.

Limits

Only Realtime Compute for Apache Flink whose engine version is vvr-4.0.12-flink-1.13 or later provides code templates.

Precautions

If a code template that is used for a draft does not contain the INSERT INTO statement, you cannot deploy the draft.

Supported code templates

The following table describes the code templates that are provided by fully managed Flink and the use scenario of each code template. You can view the supported code templates in the console of fully managed Flink.

Category

Template name

Scenario

Basic templates

Create Table

If you want to create a table and save the table in a catalog, you can execute the CREATE TABLE statement. The operation on a table is similar to the operation on a database.

Create Temporary Table

If you want to use a table only in the current session or an SQL script, you can execute the CREATE TEMPORARY TABLE statement to create a temporary table.

Create Temporary View

If you want to reuse code, create a long query, or write an SQL script to simplify draft development when you develop a Flink SQL draft, you can execute the CREATE TEMPORARY VIEW statement to create a temporary view.

INSERT INTO

If you want to write query results to a table in a downstream storage system, you can execute the INSERT INTO statement.

STATEMENT SET

If you want to write query results to two or more downstream storage systems, you can use the STATEMENT SET syntax.

Watermark

If you want fully managed Flink to process out-of-order data as expected, you can use the WATERMARK clause in DDL statements. You can use the WATERMARK clause in a CREATE TABLE DDL statement to create a watermark. This clause defines an expression to generate watermarks based on an existing field and marks the field as a time attribute field. For more information, see Event Time and Watermarks.

Aggregation and analysis templates

GROUP BY

If you want to aggregate and analyze data in real time, you can use the GROUP BY clause.

Tumble Window Aggregation

If you want to group time series data and aggregate and analyze the data in each group, you can use the TUMBLE function.

Hop Window Aggregation

If you want to update data in a window at a specified interval, you can use the HOP function.

Cumulate Window Aggregation

If you want to view the calculation results of a window in advance, such as viewing the latest result of the window at an interval of one minute, you can use the CUMULATE function.

Note

The CUMULATE function cannot be used to aggregate Change Data Capture (CDC) data. If you use the CUMULATE function to aggregate CDC data, the error message "org.apache.flink.table.api.TableException: Not supported window type: CUMULATE(max_size=[xxx ms], step=[xxx min])" appears.

Session Window Aggregation

If you want to collect the statistics of data in an active session in real time, you can use a SESSION function.

Over Window Aggregation

If you want to perform statistical analysis on each element in a window, you can use an OVER window.

Chained Window Aggregation

If you want to aggregate data of the same stream in different time dimensions, such as 1 minute, 5 minutes, 30 minutes, or 1 hour, you can use cascading windows.

Deduplication template

Deduplication

If you want to remove duplicate data from a data stream, you can use the deduplication syntax.

Top-N templates

Top-N

If you want to sort the computing results in ascending or descending order based on your business requirements and obtain the first N data records, you can use the Top-N syntax.

Window Top-N

If you want to sort the computing results in ascending or descending order and obtain the first N data records in a specified time range, you can use the window Top-N syntax.

Complex event processing (CEP) template

Pattern Recognizing

If you want to search for a set of event patterns in data streams, you can use the MATCH_RECOGNIZE syntax.

JOIN templates

Regular Join

If you want to perform a join query on data of a table and a stream table, you can use a regular join.

Interval Join

If you want to perform a join query on data of a table and a stream table in a specified period of time, you can use an interval join.

Temporal Join

If you want to perform a join query on data of a stream table and the related time version data in another table, you can use a temporal join.

Look-up Join

If you want to perform a join query on data of a static dimension table and a stream table, you can use a look-up join.

Procedure

This procedure describes how to use a code template. In this example, the Create Table template is used.

  1. Log on to the Realtime Compute for Apache Flink console.

  2. find the workspace that you want to manage and click Console in the Actions column.

  3. In the left-side navigation pane, click Development > ETL. In the upper-left corner of the SQL Editor page, click New.

  4. On the SQL Scripts tab of the New Draft dialog box, double-click the name of the desired template. You can also click the name of the desired template and click Next in the lower-right corner of the dialog box.

    image..png

  5. Configure the Name, Location, and Engine Version parameters in the Draft Configuration section based on your business requirements. In the Preview section, write SQL code.

    image..png

  6. Click Create.

    The subsequent operations are the same as the operations that are performed to develop a draft. For more information about how to develop a draft, see Develop an SQL draft.