To address the limitation that Realtime Compute for Apache Flink cannot directly access the Internet, this topic describes three connectivity options that allow you to connect your Flink workspace to the Internet, other VPCs, or an on-premises network.
Regions and zones
Before you activate Realtime Compute for Apache Flink, familiarize yourself with the following concepts:
Region
Regions are independent geographic areas where Alibaba Cloud data centers reside. A region is usually identified by the city where data centers are located. For example, the China (Hangzhou) region indicates that data centers are located in Hangzhou, China. To achieve lower network latency and faster access speed, prioritize regions geographically closer to your business data.
Zone
Zones are isolated physical areas that host data centers within a region, each with independent power and networking infrastructure. Each region typically has multiple zones. Zones in different regions are completely isolated. It is a best practice to deploy your business application in multiple zones to achieve cross-zone high availability.
Based on the location of your business data, choose optimal regions and zones that ensure high availability, stability, and low-latency data transmission.
Overview of connectivity options
The following table lists three available connectivity options. You can choose an appropriate option as needed.
Option | Scenarios |
Option | Scenarios |
Applicable to scenarios where a data source must be accessed through a public IP address, because it does not support private network connections or is not directly connected to a VPC. | |
Applicable to scenarios where a data source in another VPC must be accessed via a secure, low-latency private network connection. Cross-account or inter-region connections are supported. | |
Applicable to multi-cloud and hybrid deployment scenarios. |
Connect to the Internet
The latency of accessing a data source via the Internet is unpredictable. If your business requires low latency and high stability, we recommend that you use VPC access.
You can use Alibaba Cloud's NAT Gateway to establish a connection between a VPC and the Internet. After the connection is established, your Realtime Compute for Apache Flink workspace can access data sources via the Internet.
Step 1: Create an NAT Gateway and configure an EIP
Log on to the NAT Gateway console.
Click Create Internet NAT Gateway.
On the purchase page, set up the NAT gateway as follows:
Region, VPC, and Associate vSwitch: Choose the region, VPC, and vSwitch of your Flink workspace.
You can find the information in the management console of Realtime Compute for Apache Flink by clicking of your workspace.
Access Mode: Choose SNAT-enabled Mode.
EIP: Choose Purchase EIP. If you already have an EIP, choose Select EIP.
Line Type: Choose BGP (Multi-ISP). This configuration item only appears if you have chosen Purchase EIP in the previous step.
If your Flink workspace resides in a region outside the Chinese mainland, other line type options may be available (see the table below) when you create an EIP in the EIP console. But in the NAT Gateway console, only BGP (Multi-ISP) is available. To use an EIP with your desired line type, create an EIP in the EIP console and select the EIP when creating your NAT Gateway instance.
Click Buy Now, proceed to complete the payment, and wait for the resource configuration to complete.
(Optional) Configure SNAT.
In the Actions column of the Internet NAT Gateway instance you purchased, click Configure SNAT.
Click Create SNAT Entry.
In the SNAT Entry field, choose Specify VPC. In the Select EIP field, select an EIP from the drop-down list.
Click OK.
Step 2: Authorize access to upstream/downstream systems
So far, Realtime Compute for Apache Flink can connect to a data source via the Internet. Next, you need to authorize access from Flink by adding the EIP to the firewall rules or security group policies of upstream and downstream systems.
For example, to access your MySQL database deployed on an Alibaba Cloud Elastic Compute (ECS) instance (with an EIP already bound), add a security group policy as follows:
Go to ECS console - Instance.
Click the target ECS instance name.
Select the Security Groups tab, and click the security group's name.
Under the Security Group Details tab, select the Inbound subtab in the Access Rule section, and click Quick Add.
The Quick Add dialog box appears.
For Authorization Object, paste the EIP associated with your NAT Gateway instance in Step 1. For Port Range, select MySQL (3306).
Click OK.
Your Flink workspace can now access the MySQL database deployed on the ECS instance through the instance's public IP address.
Go to the development console of Realtime Compute for Apache Flink. Click the Network detection icon in the upper-right corner to test network connectivity.
Connect to other VPCs
If other services are in the early planning stage or the cost of replacement is low, we recommend that you relaunch your resources into the same VPC as your Flink workspace. Alternatively, you can release your current Flink workspace and create a new one in the same VPC as other services.
You can use one of the following methods to establish cross-VPC network connections:
VPC peering connections: A VPC peering connection is a networking connection between two VPCs that enable them to communicate through private IP addresses, as if they were in the same network. You can create a VPC peering connection between your own VPCs, or with a VPC in another Alibaba Cloud account. Inter-region VPC peering connections are also supported.
Transit routers: A transit router can be used to connect network instances and forward traffic between them in the same region or across regions. After a transit router is connected with your VPC, routes are automatically synchronized. Only one transit router is allowed per region. To create inter-region connections, deploy a transit router in each region.
PrivateLink: PrivateLink allows you to access resources in a VPC from another VPC over secure and private networks. PrivateLink simplifies network architecture and avoids security risks from the Internet.
Item | VPC peering connections | Transit routers | PrivateLink |
Item | VPC peering connections | Transit routers | PrivateLink |
Connection method | VPCs are connected in pairs. | VPCs are connected through a transit router. | A unidirectional connection is established between VPCs through the endpoint service. |
Support for route propagation | Not supported | Supported | Not supported |
Bidirectional or unidirectional connection | Bidirectional connection | Bidirectional connection | Unidirectional connection |
Support for cross-account resource access | Supported | Supported | Supported |
Support for inter-region access | Supported | Supported | Not supported |
Suitable scenarios | Interconnecting a few VPCs | Interconnecting many VPCs | Establishing a unidirectional connection from one VPC to another |
Configuration complexity | High. You need to establish a peering connection between every VPC pair and update the route tables of both VPCs. | Low. You need to connect your VPC to a transit router and configure the VPC route table to direct traffic to the transit router. | Low. You can connect VPCs that have overlapping CIDR blocks and do not need to configure route tables. |
Network latency | Low-latency | Medium-latency. The network latency is higher because a transit router adds an additional hop between VPCs. | Low-latency |
Cost |
|
| You are charged for the PrivateLink service on a pay-as-you-go basis, including instance fees and data transfer fees. |
Support for overlapping CIDR blocks | Not supported | Not supported | Supported |
Example
A company has created VPC A in the China (Hangzhou) region and VPC B in the China (Beijing) region. Its Realtime Compute for Apache Flink workspace is deployed in the China (Hangzhou) region, while ECS instances for data storage and development are deployed in the China (Beijing) region.
A VPC peering connection is created between VPC A and B.
VPCs and vSwitches involved in peering must not have overlapping CIDR blocks.
For example, if VPC A has a CIDR block of 192.168.0.0/16 and VPC B has a CIDR block of 192.168.0.0/24, they cannot be connected, even if a peering connection is created.
To peer VPCs across Alibaba Cloud accounts, ensure that both the requester and accepter accounts have VPCs.
For detailed instructions, see Use VPC peering connection for private communication.
Connect to on-premises networks
To connect to on-premises data centers, use any of the following services:
Express Connect: Express Connect can connect on-premises data centers or other cloud platforms to Alibaba Cloud through Express Connect circuits. Express Connect offers low-latency, low-packet-loss, and high-bandwidth connections, even when data is transmitted over a long distance.
VPN Gateway: VPN Gateway provides network connection services that securely and reliably connect data centers, office networks, and Internet clients to Alibaba Cloud through encrypted and private tunnels.
The table below compares the two services:
Item | Express Connect | VPN Gateway |
Item | Express Connect | VPN Gateway |
Quality | High (over Express Connect circuits) | Low (over the Internet) |
Implementation period | Long (2 to 3 months) | Short (out-of-the-box) |
Cost | High | Low |
Bandwidth | A single Express Connect circuit supports a maximum bandwidth of 100 Gbps. Multiple Express Connect circuits can achieve Tbps-level bandwidth. | The bandwidth is limited by the bandwidth limit of your public IP address. |
Suitable scenarios | Express Connect is suitable for financial organizations or government agencies to access cloud services securely. | VPN Gateway is ideal for moving basic services to the cloud, such as office networks or data storage systems. |
