This topic describes the basic concepts used in Realtime Compute for Apache Flink.
Hierarchy
The following figure shows how the basic concepts in Realtime Compute for Apache Flink are organized. This figure serves as a helpful reference for planning future operations, such as draft development, deployment creation, deployment O&M, and security management.
Terms
Term | Description | References |
workspace | A workspace is the basic management unit for namespaces of fully managed Flink. Workspaces are independent environments and have their own dedicated computing resources. | |
namespace | A namespace is the basic management unit for drafts and deployments of fully managed Flink. The configurations, drafts, deployments, and permissions of each namespace are managed individually. You can create multiple namespaces and assign separate resources and permissions to each of them. This effectively isolates the resources and permissions among the namespaces of different tenants. | |
resource | Fully managed Flink bills you based on compute units (CUs), which is a unit of measurement for computing resources. One CU provides computing capabilities equivalent to an hour of 1 CPU core, 4 GiB of memory, and 20 GB of local storage. The local storage stores information, such as logs and checkpoints. The number of CUs consumed reflects the resources used by Realtime Compute for Apache Flink. The number of CUs consumed by a Realtime Compute for Apache Flink deployment varies based on the queries per second (QPS) of input data streams, computing complexity, and input data distribution of the deployment. You can estimate the number of CUs that you need to purchase based on the scale of your business and the required computing capability of Realtime Compute for Apache Flink. | |
draft | You can create a draft on the SQL Editor page of the Realtime Compute for Apache Flink console. Drafts can be developed only in the Realtime Compute for Apache Flink console and cannot be developed by using SDK. | |
deployment | Deployments are used to isolate environments, such as the development and production environments. After you publish a draft, the draft is converted into a deployment. If you modify the configuration of the draft, the running deployment for the draft is not affected. The drafts that you create on the SQL Editor page can be published as deployments. The JAR packages and Python packages that you upload can be used to create deployments. A deployment has deterministic streaming and batch properties. Deployments can be developed in the Realtime Compute for Apache Flink console or by using SDKs. | |
job | A job is an instance of a workload that runs in a deployment. A job has deterministic streaming and batch properties. | N/A |
connector | Realtime Compute for Apache Flink provides a variety of built-in connectors to read data from or write data to upstream and downstream data stores and perform data synchronization. Realtime Compute for Apache Flink also allows you to upload and use JAR packages of custom connectors. | |
function | Realtime Compute for Apache Flink provides functions, including built-in functions and user-defined functions (UDFs). | |
catalog | Metadata management is one of the most critical aspects of data processing. The catalogs of Realtime Compute for Apache Flink contain metadata, such as databases, tables, fields, partitions, and the data stored in databases or other external systems. | |
role | A role is a collection of users who have the same permissions. This concept is used in the security features of Realtime Compute for Apache Flink. One user can assume multiple roles, and multiple users can assume the same role. After you grant permissions to a role, all users who are assigned this role are granted the same permissions. | |
user | An Alibaba Cloud account or a RAM user can be added as a user of Realtime Compute for Apache Flink. This concept is used in the security features of Realtime Compute for Apache Flink. A user can manage data, drafts, deployments, resources, and functions in a namespace of Realtime Compute for Apache Flink only after the user is added to the namespace and is granted the related permissions on the namespace. | |
queue | A queue can be specified for deployments to implement resource isolation and management. |