Container Service for Kubernetes:deploy multiple ingress controllers

Notes

Below are notes on configuring multiple Ingress controllers, including setting a unique IngressClass and adhering to Service name length restrictions.

• Set up a unique IngressClass

  When operating multiple Ingress controllers, it's essential to assign a unique IngressClass to each to ensure smooth functioning and prevent conflicts. The parameters to be modified are as follows:

  • controller.ingressClassResource.name: Assign a unique name to each Ingress controller.

  • controller.ingressClassResource.controllerValue: Define a unique identifier for each Ingress controller.

• Restrictions on Service Name Length

  When you deploy the ack-ingress-nginx-v1 Helm application and use ack-ingress-nginx-v1 as the application name, the resulting Service will be named <application name>-ack-ingress-nginx-v1-controller. For an internal type LoadBalancer Service, the name will be <application name>-ack-ingress-nginx-v1-controller-internal. Make sure these names do not exceed 63 characters to avoid resource creation failures and potential error messages.

  no service with name xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal found in namespace open-api-test-inter: services "xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal" not found

  Should you encounter this error, verify that the Helm application name is short enough and then proceed with redeployment.

Deploy a new NGINX Ingress controller service

Upon creation of an ACK cluster, an NGINX Ingress controller with two pods is automatically deployed, along with an Internet-facing SLB instance as the frontend load balancing Service.

Follow these steps to deploy a completely independent NGINX Ingress controller service within the ACK cluster.

1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

2. On the Cluster List page, click the desired cluster name. Then, in the left-side navigation pane, select Applications > Helm.

3. On the Helm page, click Deploy. In the Basic Information step, configure the parameters based on the following table.

   Parameter	Example Value
   Application Name	ack-ingress-nginx Note You can customize it, and pay attention to the Service name length restriction.
   Namespace	kube-system
   Source	Default is App Market
   Chart	Scenarios: Select All. Supported Architecture: Select amd64. Search Box: Search for Ack-ingress-nginx. Select ack-ingress-nginx for clusters of version 1.20 and below. Select ack-ingress-nginx-v1 for clusters of version 1.22 and above.

   Choose the Chart name that matches your cluster version and click Next.

4. On the Parameter Configuration page, select Chart Version , and then click OK.

   Note

   Chart version 4.0.17 and later (ack-ingress-nginx-v1 version 1.8.0-aliyun.1 and later) are compatible only with Kubernetes 1.22 or newer. If your cluster runs Kubernetes 1.20, opt for chart version 4.0.16 (ack-ingress-nginx-v1 version 1.2.1-aliyun.1).

   The primary parameters for ack-ingress-nginx-v1 are listed below.

   Parameter	Description
   controller.image.repository	The image repository of ingress-nginx.
   controller.image.tag	The image version of ingress-nginx. For more information, see Nginx Ingress Controller.
   controller.ingressClassResource.name	Set the name of the IngressClass corresponding to the Ingress controller. Important This parameter serves as a replacement for the controller.ingressClass parameter in templates for versions below 1.22 and can be used normally in the kubernetes.io/ingress.class annotation. The IngressClass names created by different sets of Ingress controllers in the same cluster must be unique and cannot be set to the nginx keyword (nginx is the listening identifier of the default Ingress controller in the cluster).
   controller.ingressClassResource.controllerValue	Set the Controller Class corresponding to the Ingress controller. Important The Controller Class created by different sets of Ingress controllers in the same cluster must be unique and cannot be set to the k8s.io/ingress-nginx keyword (k8s.io/ingress-nginx is the listening identifier of the default Ingress controller in the cluster).
   controller.replicaCount	Set the number of replicas for the Ingress controller pod.
   controller.service.enabled	Specifies whether to enable SLB access (including public and private networks).
   controller.service.external.enabled	Specifies whether to enable Internet-facing SLB access. If you do not need to enable it, set it to false.
   controller.service.internal.enabled	Specifies whether to enable internal-facing SLB access. If you need to enable it, set it to true.
   controller.kind	Set the deployment pattern of the Ingress controller. Optional values: Deployment and DaemonSet.
   controller.electionID	The ID used for leader election to update the Ingress endpoint status. Important If you deploy multiple sets of NGINX Ingress controllers in the same namespace through the App Market, you need to set the electionID to different values to avoid leader election conflicts.
   controller.metrics.enabled	Specifies whether to enable the ingress-nginx metrics.
   controller.metrics.serviceMonitor.enabled	Specifies whether to enable the ServiceMonitor, which provides the configuration on how metrics are collected. Note We recommend that you enable the ServiceMonitor after you enable the ingress-nginx metrics. Then, the system automatically generates the Prometheus configuration to collect metrics.

5. View the NGINX Ingress controller service that has been deployed.

   Return to the Helm page. You will notice the new NGINX Ingress controller service has been successfully deployed.

Test network connectivity

Deploy a test application and configure it to use the newly deployed NGINX Ingress controller for service exposure.

1. Deploy an NGINX test application.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nginx
   spec:
     replicas: 1
     selector:
       matchLabels:
         run: nginx
     template:
       metadata:
         labels:
           run: nginx
       spec:
         containers:
         - image: nginx
           imagePullPolicy: Always
           name: nginx
           ports:
           - containerPort: 80
             protocol: TCP
         restartPolicy: Always
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: nginx
   spec:
     ports:
     - port: 80
       protocol: TCP
       targetPort: 80
     selector:
       run: nginx
     sessionAffinity: None
     type: NodePort

2. Configure the Ingress for Internet-facing service access.

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: nginx
     annotations:
       # Note that you need to set this to the INGRESS_CLASS you configured earlier.
       kubernetes.io/ingress.class: "<YOUR_INGRESS_CLASS>"
   spec:
     rules:
     - host: foo.bar.com
       http:
         paths:
         - path: /
           backend:
             service: 
               name: nginx
               port:
                 number: 80
           pathType: ImplementationSpecific

   Note

   You need to configure the annotation kubernetes.io/ingress.class.

   After deploying the application, you can carry out the following steps to view the IP addresses for both the Ingress resource's endpoint and the newly deployed NGINX Ingress controller service.

   • To query the IP address of the default nginx-ingress-lb Service's associated Internet-facing SLB instance, execute the command below.

      kubectl -n kube-system get svc nginx-ingress-lb

     You should see the following result:

     NAME               TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)                      AGE
     nginx-ingress-lb   LoadBalancer   172.19.XX.XX   192.0.XX.XX   80:31429/TCP,443:32553/TCP   2d

   • To query the IP address of the newly created nginx-ingress-lb Service's associated Internet-facing SLB instance, execute the command below.

     kubectl -n <YOUR_NAMESPACE> get svc nginx-ingress-lb

     You should see the following result:

     NAME               TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                      AGE
     nginx-ingress-lb   LoadBalancer   172.19.XX.XX   198.51.XX.XX   80:30969/TCP,443:31325/TCP   39m

   • To query the application's Ingress configuration, execute the command below.

     kubectl get ing

     You should see the following result:

     NAME      HOSTS         ADDRESS         PORTS     AGE
     nginx     foo.bar.com   198.51.XX.XX   80        5m

   The output above indicates that the Ingress IP address matches the IP address of the newly deployed NGINX Ingress controller.

3. Test access to the application through both the default and new NGINX Ingress controllers.

    # Access the application through the default NGINX Ingress controller. The expected result is 404.
     curl -H "Host: foo.bar.com" http://192.0.2.0
   default backend - 404                                                                                                                                                                                        
     # Access the application through the new NGINX Ingress controller. The NGINX welcome page is expected.
     curl -H "Host: foo.bar.com" http://198.51.XX.XX
   <!DOCTYPE html>
   <html>
   <head>
   <title>Welcome to nginx!</title>
   <style>
       body {
           width: 35em;
           margin: 0 auto;
           font-family: Tahoma, Verdana, Arial, sans-serif;
       }
   </style>
   </head>
   <body>
   <h1>Welcome to nginx!</h1>
   <p>If you see this page, the nginx web server is successfully installed and
   working. Further configuration is required.</p>
   
   <p>For online documentation and support please refer to
   <a href="http://nginx.org/">nginx.org</a>.<br/>
   Commercial support is available at
   <a href="http://nginx.com/">nginx.com</a>.</p>
   
   <p><em>Thank you for using nginx.</em></p>
   </body>
   </html>

The test results indicate that services exposed via separate NGINX Ingress controllers operate independently. This setup is ideal for scenarios in which certain services within the same cluster require exposure to the internet, while others are intended solely for internal access by services within the same VPC but external to the Kubernetes cluster.