Manage multi-cluster gateways - Container Service for Kubernetes

This topic describes how to enable and disable the multi-cluster gateway feature on a Distributed Cloud Container Platform for Kubernetes (ACK One) Fleet instance and how to connect associated clusters to a multi-cluster gateway.

Billing

You are charged for using multi-cluster gateways. For more information about the billing of multi-cluster gateways, see Billing overview.

Prerequisites

Microservices Engine (MSE) cloud-native gateway authorization is complete.

The AliyunAdcpManagedMseRolePolicy policy is attached to the AliyunAdcpManagedMseRole Resource Access Management (RAM) role. The following code block shows the policy document of AliyunAdcpManagedMseRolePolicy.

Policy document of AliyunAdcpManagedMseRolePolicy

{
    "Statement": [
        {
            "Action": [
                "mse:AddBlackWhiteList",
                "mse:AddGateway",
                "mse:AddServiceSource",
                "mse:CreateApplication",
                "mse:DeleteGateway",
                "mse:DeleteServiceSource",
                "mse:GetBlackWhiteList",
                "mse:GetGateway",
                "mse:GetGatewayDetail",
                "mse:GetGatewayOption",
                "mse:ListServiceSource",
                "mse:ListTagResources",
                "mse:ModifyLosslessRule",
                "mse:TagResources",
                "mse:UntagResources",
                "mse:UpdateBlackWhiteList",
                "mse:UpdateGatewayOption",
                "mse:UpdateServiceSource"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "log:CloseProductDataCollection",
                "log:OpenProductDataCollection",
                "log:GetProductDataCollection"
            ],
            "Resource": [
                "acs:mse:*:*:instance/*",
                "acs:log:*:*:project/*/logstore/mse_*"
            ],
            "Effect": "Allow"
        },
        {
            "Action": "ram:CreateServiceLinkedRole",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "ram:ServiceName": [
                        "audit.log.aliyuncs.com"
                    ]
                }
            },
            "Effect": "Allow"
        }
    ],
    "Version": "1"
}

The Fleet management feature is enabled. The kubeconfig file of the Fleet instance is obtained in the ACK One console and a kubectl client is connected to the Fleet instance.
Associated clusters are added. The associated clusters and the ACK One Fleet instance are deployed in the same virtual private cloud (VPC).
The latest version of Alibaba Cloud CLI is installed and configured.

Enable the multi-cluster gateway feature

Use the console

Log on to the ACK One console. In the left-side navigation pane, choose Fleet > Multi-cluster Gateways.
On the Multi-cluster Gateway page, click Enable Multi-cluster Gateway. In the message that appears, click OK.

Use the CLI

Make sure that Alibaba Cloud CLI is up-to-date and run the following command to enable the multi-cluster gateway feature.
Replace <your_fleet_clusterid> with the ID of your Fleet instance.
```
aliyun adcp UpdateHubClusterFeature --ClusterId <your_fleet_clusterid> --GatewayEnabled true
```

Run the following command to check whether the multi-cluster gateway feature is enabled:

aliyun adcp DescribeHubClusterDetails --ClusterId <your_fleet_clusterid> |grep -B4 -A1 EnabledMSE`

Expected output: If the value of the Status parameter is True, the multi-cluster gateway feature is enabled.

			{
				"Message": "",
				"Reason": "",
				"Status": "True",
				"Type": "EnabledMSE"
			},

Create a multi-cluster gateway

Use the console

Log on to the ACK One console. In the left-side navigation pane, choose Fleet > Multi-cluster Gateways.
In the upper-right corner of the Multi-cluster Gateway page, click Create Gateway.
In the panel that appears, modify the YAML file that is used to create the multi-cluster gateway based on your business requirements and click Create.

Use the CLI

Obtain and record the vSwitch ID of the ACK One Fleet instance.
1. Run the following command to query the vSwitch ID:
```
aliyun adcp DescribeHubClusterDetails --ClusterId <your_fleet_clusterid>
```
1. Record the vSwitch ID in the VSwitches field of the output.

Create a file named mseingressconfig.yaml and add the following content to the file.

Replace ${vsw-id1} with the vSwitch ID that you recorded. You can add an annotation to the gateway configuration file to specify the associated clusters that you want to add to the gateway.

apiVersion: mse.alibabacloud.com/v1alpha1
kind: MseIngressConfig
metadata:
  name: ackone-gateway
  # Connect associated clusters to the MSE gateway. 
  #annotations:
  #  mse.alibabacloud.com/remote-clusters: ${cluster1},${cluster2}
spec:
  common:
    instance:
      replicas: 3
      spec: 2c4g
    network:
      # You can configure both an Internet-facing Server load Balancer (SLB) instance and an internal-facing SLB instance. If no SLB instance is specified, an Internet-facing SLB instance is used by default. 
      #publicSLBSpec: slb.s2.small
      #privateSLBSpec: slb.s2.small
      vSwitches:
      - ${vsw-id1}
  ingress:
    local:
      ingressClass: mse
  name: mse-ingress

Run the following command to create a gateway named mse-ingress on the ACK One Fleet instance:
```
kubectl apply -f mseingressconfig.yaml
```
Run the following command to check whether the gateway is created:
```
kubectl get mseingressconfig ackone-gateway
```
Expected output:
```
NAME             STATUS      AGE
ackone-gateway   Listening   3m15s
```
The output indicates that the gateway is in the Listening state. This means that the cloud-native gateway is created and running. The gateway listens on Ingresses whose IngressClasses are mse.
The status of a gateway created from an MseIngressConfig changes in the following order: Pending, Running, and Listening. State description:
- Pending: The cloud-native gateway is being created. This process may take about 3 minutes.
- Running: The cloud-native gateway is created and running.
- Listening: The cloud-native gateway is running and listens on Ingresses.
- Failed: The cloud-native gateway is invalid. You can check the message in the Status field to troubleshoot the issue.

Add or remove associated clusters

Use the console

Log on to the ACK One console. In the left-side navigation pane, choose Fleet > Multi-cluster Gateways.
In the upper part of the Multi-cluster Gateway page, select the multi-cluster gateway that you want to manage from the Select a gateway drop-down list and click Modify in the upper-right corner.
In the ModifyGateway panel, modify the MseIngresConfig.yaml file. Modify the cluster IDs in the annotations parameter and click Update.
Sample code:
```
annotations:
  mse.alibabacloud.com/remote-clusters: ${cluster1-id},${cluster2-id}
```
- ${cluster1-id} and ${cluster2-id} are the IDs of the associated clusters. Separate multiple cluster IDs with commas (,). You can modify the cluster IDs to add or remove associated clusters.
- If you do not add associated clusters to a multi-cluster gateway when you create the multi-cluster gateway, the MseIngresConfig.yaml file does not have the annotations parameter. To add specified associated clusters to the multi-cluster gateway, you must add the preceding code to the metadata object in the MseIngresConfig.yaml file and then modify the cluster IDs.

Use the CLI

You can modify the corresponding annotation in the mseingressconfig object of the ACK One Fleet instance to add or remove associated clusters. Replace ${cluster1-id} and ${cluster2-id} with the IDs of the associated clusters. Separate multiple cluster IDs with commas (,).
```
annotations:
  mse.alibabacloud.com/remote-clusters: ${cluster1-id},${cluster2-id}
```
Run the following command to check whether the associated clusters are added to the multi-cluster gateway:
```
kubectl get mseingressconfig ackone-gateway -ojsonpath="{.status.remoteClusters}"
```
Expected output:
```
[{"clusterId":"c7fb82****"},{"clusterId":"cd3007****"}]
```
The output indicates the IDs of the associated clusters and that no failure message is returned. This means that the associated clusters are added to the multi-cluster gateway.
Run the following command to query the associated clusters that are added to the multi-cluster gateway:
```
kubectl get mseingressconfig ackone-gateway
```
Run the following command to add more associated clusters or remove associated clusters.
mse.alibabacloud.com/remote-clusters=${cluster1},${cluster2} indicates the associated clusters that you want to add to the multi-cluster gateway.
```
kubectl annotate mseingressconfig ackone-gateway mse.alibabacloud.com/remote-clusters=${cluster1},${cluster2} --overwrite=true
```

Disable the multi-cluster gateway feature

Important

Deleting multi-cluster gateways can adversely affect your businesses. Proceed with caution.
If you no longer need multi-cluster gateways, you can delete the multi-cluster gateways and then disable the multi-cluster gateway feature to avoid incurring unexpected fees.

Use the console

Log on to the ACK One console. In the left-side navigation pane, choose Fleet > Multi-cluster Gateways.
In the upper part of the Multi-cluster Gateway page, select the multi-cluster gateway that you want to manage from the Select a gateway drop-down list and click Delete in the upper-right corner.
In the dialog box that appears, enter the name of the multi-cluster gateway that you want to delete and click Delete.
Confirm that all multi-cluster gateways within the ACK One Fleet instance are deleted and click Disable in the upper-right corner of the page.
In the message that appears, click OK.

Use the CLI

Run the following command to delete a multi-cluster gateway:
```
kubectl delete mseingressconfig ackone-gateway
```

Run the following command to disable the multi-cluster gateway feature:

aliyun adcp UpdateHubClusterFeature --ClusterId <your_fleet_clusterid> --GatewayEnabled false