A hybrid cluster connects your on-premises Kubernetes cluster with Alibaba Cloud by registering an external Kubernetes cluster with Container Service for Kubernetes (ACK) and adding Elastic Compute Service (ECS) nodes to it. The cluster then manages both cloud and on-premises computing resources under a unified control plane.
This topic explains how to choose a network mode for a hybrid cluster and how to connect the cloud network to the on-premises network using Express Connect.
Prerequisites
Before you begin, make sure that:
Your on-premises network meets the connectivity quality required for a reliable hybrid cluster.
You have an existing external Kubernetes cluster deployed in a data center and registered with ACK as a registered cluster.
You have an Alibaba Cloud account with permissions to create Express Connect circuits, virtual border routers (VBRs), and Cloud Enterprise Network (CEN) instances.
Choose a network mode
The right network mode depends on your cluster size and network performance requirements. The following table compares the available options.
| Network mode | Networking model | When to use |
|---|---|---|
| Flannel VXLAN | Overlay | Clusters with fewer than 100 nodes; no high network performance requirement |
| Calico IPIP | Overlay | Clusters with fewer than 100 nodes; no high network performance requirement |
| Cilium VXLAN | Overlay | Clusters with fewer than 100 nodes; no high network performance requirement |
| Calico route reflection | BGP routing | Large clusters or clusters requiring high network performance |
| Cilium Border Gateway Protocol (BGP) routing | BGP routing | Large clusters or clusters requiring high network performance |
Most external Kubernetes clusters use Calico routing mode. This topic uses Calico route reflection as the example configuration. For the cloud network, ACK provides the Terway plug-in for container network management.
Calico plug-in runs only in the on-premises network. Terway plug-in runs only in the cloud network. For details on deploying Terway, see Deploy and configure Terway.
How it works
The following figure shows the network topology of a hybrid cluster connecting an on-premises data center to the Alibaba Cloud network.
In this example configuration:
On-premises network uses Calico route reflection mode
Cloud network uses the One ENI for Multi-Pod mode of Terway
CIDR blocks used in this example:
| Network | CIDR |
|---|---|
| Data center private CIDR | 192.168.0.0/24 |
| Container network CIDR | 10.100.0.0/16 |
| Virtual private cloud (VPC) CIDR | 10.0.0.0/8 |
| vSwitch for compute nodes | 10.10.24.0/24 |
| vSwitch for pods | 10.10.25.0/24 |
Connect the cloud network to the on-premises network
To connect cloud compute nodes and pods to their on-premises counterparts, use Express Connect to establish a dedicated private connection between your data center and Alibaba Cloud. The following figure shows how nodes and pods are connected.
For more information, see Connect a data center to ECS by using an Express Connect circuit.
Use an Express Connect circuit to connect the on-premises network to Alibaba Cloud. For information about setting up the physical connection, see Physical Connection.
Create a connection over the Express Connect circuit to link edge devices in the data center to a virtual border router (VBR) that acts as the gateway on the cloud side.
Attach the VBR and VPC to a Cloud Enterprise Network (CEN) instance.
Configure BGP on the VBR and in the data center. For detailed BGP configuration steps, see Configure BGP networks for data centers.
Test network connectivity between the cloud network and the on-premises network.
Configure routes that point to the private CIDR blocks used by cloud services to communicate with the on-premises network:
Container Registry: Add routes that point to the private address of ACK component images
Object Storage Service (OSS): Internal endpoints of OSS buckets and VIP ranges
What's next
After connecting the networks, deploy and configure Terway on the cloud nodes to complete the hybrid cluster network setup: