Container Service for Kubernetes:Overview of hybrid networks

Network mode of hybrid clusters

You can add ECS nodes to an external Kubernetes cluster that is registered with ACK to create a hybrid cluster and use the cluster to connect cloud networks to on-premises networks. To do this, you first need to set the network mode of the hybrid cluster. Design the network of the cluster nodes that are deployed in the data center based on your business requirements.

• If the external Kubernetes cluster contains less than 100 nodes, which is considered as a medium- and small-sized cluster, and the cluster does not require high network performance, you can select one of the following network modes:

  • Flannel VXLAN

  • Calico IPIP

  • Cilium VXLAN

• If the size of the external Kubernetes cluster is large or you want to create a large hybrid cluster and the cluster requires high network performance, you can select one of the following network modes:

  • Calico route reflection

  • Cilium Border Gateway Protocol (BGP) routing

In most cases, external Kubernetes clusters use the Calico routing mode. This topic provides an example on how to configure a Kubernetes cluster that is deployed in a data center to use the Calico routing mode. For container network plug-ins, we recommend that you choose a custom network plug-in provided by the cloud platform that you use. ACK provides the Terway plug-in to help you manage container networks. The following figure shows the networking of a hybrid cluster.

The private CIDR block of the data center is 192.168.0.0/24 and the CIDR block of the container network is 10.100.0.0/16. The on-premises network uses the Calico route reflection mode. The CIDR block of the virtual private cloud (VPC) is 10.0.0.0/8, the CIDR block of the vSwitch for compute nodes is 10.10.24.0/24, and the CIDR block of the vSwitch for pods is 10.10.25.0/24. The cloud network uses the One ENI for Multi-Pod mode of Terway.

To create a hybrid network, you must connect the cloud network to the on-premises network by performing the following operations:

• Connect the on-premises network to the VPC.

• Connect the on-premises container network to the container network in the cloud.

Connect the cloud network to the on-premises network

To connect a cloud network to an on-premises network in cloud-native scenarios, you need to connect compute nodes and pods that are deployed in both networks. The following figure shows how nodes and pods are connected.

To connect the cloud network to the on-premises network, perform the following steps. For more information, see Connect a data center to ECS by using an Express Connect circuit.

1. Use an Express Connect circuit to connect the on-premises network to Alibaba Cloud.

   For more information about the corresponding solution, see Physical Connection.

2. Create a connection over an Express Connect circuit to connect edge devices in the data center to a virtual border router (VBR) that functions as a gateway in the cloud.

3. Attach the VBR and VPC to a Cloud Enterprise Network (CEN) instance.

4. Configure BGP on the VBR and in the data center.

5. Test the network connectivity between the cloud network and on-premises network.

6. Configure routes that point to the private CIDR blocks used by the cloud services to communicate with the on-premises network. For more information about the operations, see the following topics:

   • Container Registry: Add routes that point to the private address if ACK component images

   • Managed Service for Prometheus: VPC endpoints and the corresponding CIDR blocks of Managed Service for Prometheus.

   • Object Storage Service (OSS): Internal endpoints of OSS buckets and VIP ranges.