Implement hybrid disaster recovery based on an ALB multi-cluster gateway in ACK One - Server Load Balancer

To build an active-zone redundancy system to improve the availability of businesses deployed in Kubernetes clusters running in data centers or third-party platforms, you can use Distributed Cloud Container Platform for Kubernetes (ACK One) to centrally manage traffic, applications, and clusters. ACK One can be used to route traffic across clusters and seamlessly perform failovers. This topic describes how to use ACK One to quickly build an active zone-redundancy system in a hybrid cloud environment.

Architecture

The preceding figure shows the architecture of an active zone-redundancy system in a hybrid cloud environment. The system consists of a registered cluster, a Fleet instance (GitOps is optional), and an Application Load Balancer (ALB) multi-cluster gateway.

Resources on Alibaba Cloud are deployed in the same virtual private cloud (VPC). A Container Service for Kubernetes (ACK) cluster is created in AZ 1 and a registered cluster is created in AZ 2.
Connect a Kubernetes cluster deployed in a data center or on a third-party platform to the registered cluster. Use a leased line to connect the data center to the VPC.
Associate the ACK cluster and registered cluster with the Fleet instance in the same VPC. Use ACK One GitOps to distribute an application to the ACK cluster and registered cluster.
On the Fleet instance, configure an AlbConfig to create an ALB multi-cluster gateway and add clusters to the gateway. Then, create an Ingress and configure traffic routing rules on the Fleet instance to manage north-south traffic and implement zone-disaster recovery.

Procedure

Step 1: Design the network and create clusters

Create a Fleet instance, ACK cluster, and registered cluster in the same VPC. Make sure that the ACK cluster and registered cluster reside in different zones.
Make sure that the node and pod CIDR blocks of the ACK cluster do not overlap with those of the Kubernetes cluster. For more information about network design for Fleet management, see Network design for Fleet management.

Step 2: Connect to Alibaba Cloud

Connect the Kubernetes cluster to the registered cluster. For more information, see Create a registered cluster.
Note
- To use the registered cluster to migrate workloads to Alibaba Cloud with elastic resources, see Build a hybrid cloud cluster and add ECS instances to the cluster and Schedule pods to elastic container instances that are deployed as virtual nodes.
- To withstand unexpected traffic spikes, you can configure high availability settings for elastic container instances. For more information, see Create ECIs across zones.
Connect the on-premises network to the VPC. For more information, see Network connectivity.
Use a leased line to connect to Alibaba Cloud. For more information, see Overview of hybrid networks.
For more information about how to associate the registered cluster and ACK cluster with the Fleet instance, see Manage associated clusters.

Step 3: Use a GitOps ApplicationSet to distribute an application to multiple clusters

In a hybrid cloud environment, you must confirm which container network is used the Kubernetes cluster: underlay or overlay. The difference between the two container networks is that you must connect the on-premises network to the VPC if you want to access pods in the overlay network. In addition, you must ensure that each node can route traffic to the pod CIDR block in the on-premises network. An ALB multi-cluster gateway supports both container networks. You can use ClusterIP Services in the underlay network but must use NodePort Services in the overlay network.

The following example shows the YAML templates of the GitOps ApplicationSet named web-demo for the underlay and overlay networks. For more information about ApplicationSets, see Create a multi-cluster application.

Underlay container network

Use the following YAML template to deploy the web-demo ApplicationSet. ClusterIP Services are used in all associated clusters.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: appset-web-demo-undelay
  namespace: argocd
spec:
  template:
    metadata:
      name: '{{.metadata.annotations.cluster_id}}-web-demo'
      namespace: argocd
    spec:
      destination:
        name: '{{.name}}'
        namespace: gateway-demo
      project: default
      source:
        repoURL: https://github.com/AliyunContainerService/gitops-demo.git
        path: manifests/helm/web-demo
        targetRevision: main
        helm:
          valueFiles:
            - values.yaml
          parameters:
            - name: envCluster
              value: '{{.metadata.annotations.cluster_name}}'
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true
  generators:
    - clusters:
        selector:
          matchExpressions:
            - values:
                - cluster
              key: argocd.argoproj.io/secret-type
              operator: In
            - values:
                - in-cluster
              key: name
              operator: NotIn
  goTemplateOptions:
    - missingkey=error
  syncPolicy:
    preserveResourcesOnDeletion: false
  goTemplate: true

Overlay container network

Access the ArgoCD UI, choose Settings > Clusters, and add the cluster: ack and cluster: idc labels to the two clusters.
Use the following YAML template to deploy the web-demo ApplicationSet in the two clusters. Use NodePort Services in the data center and use ClusterIP Services in the ACK cluster.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: appset-web-demo-overlay
  namespace: argocd
spec:
  template:
    metadata:
      name: '{{.metadata.annotations.cluster_id}}-web-demo-overlay'
      namespace: argocd
    spec:
      destination:
        name: '{{.name}}'
        namespace: gateway-demo
      project: default
      source:
        repoURL: https://github.com/AliyunContainerService/gitops-demo.git
        path: manifests/helm/web-demo
        targetRevision: main
        helm:
          valueFiles:
            - values.yaml
          parameters:
            - name: isNodePort
              value: "{{.values.isNodePort}}"
            - name: envCluster
              value: '{{.metadata.annotations.cluster_name}}'
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true
  generators:
    - clusters:
        selector:
          matchLabels:
            cluster: 'idc'
        # A key-value map for arbitrary parameters
        values:
          isNodePort: "true"
    - clusters:
        selector:
          matchLabels:
            cluster: 'ack'
        values:
          isNodePort: "false"
  goTemplateOptions:
    - missingkey=error
  syncPolicy:
    preserveResourcesOnDeletion: false
  goTemplate: true

Step 4: Use the ALB multi-cluster gateway to implement active zone-redundancy

Configure an AlbConfig on the Fleet instance to create an ALB multi-cluster gateway, and add the ACK cluster and registered cluster to the gateway.
Create routing rules and Ingresses on the Fleet instance to implement active zone-redundancy.

For more information, see Build a zone-disaster recovery system.