All Products
Search
Document Center

Container Service for Kubernetes:deploy multiple ingress controllers

Last Updated:Feb 10, 2025

This topic explains how to deploy multiple independent NGINX Ingress controllers in an Alibaba Cloud Container Service for Kubernetes (ACK) cluster, enabling differentiated service access.

Background information

You can configure both public and private network types for NGINX Ingress controllers to modify the default configuration of the NGINX Ingress controller in an ACK cluster for use with an internal-facing SLB instance. The document describes two modes that cover most use cases. However, in certain situations, such as when cluster-based public network services need to be exposed via a public network Ingress for external access, while private network services should only be accessible within the same VPC and not from the public network, you can set up two separate NGINX Ingress controller services. Each service would have a frontend connected to a different network type of SLB instance, fulfilling the requirements of these specific scenarios.

Notes

Below are notes on configuring multiple Ingress controllers, including setting a unique IngressClass and adhering to Service name length restrictions.

  • Set up a unique IngressClass

    When operating multiple Ingress controllers, it's essential to assign a unique IngressClass to each to ensure smooth functioning and prevent conflicts. The parameters to be modified are as follows:

  • Restrictions on Service Name Length

    When you deploy the ack-ingress-nginx-v1 Helm application and use ack-ingress-nginx-v1 as the application name, the resulting Service will be named <application name>-ack-ingress-nginx-v1-controller. For an internal type LoadBalancer Service, the name will be <application name>-ack-ingress-nginx-v1-controller-internal. Make sure these names do not exceed 63 characters to avoid resource creation failures and potential error messages.

    no service with name xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal found in namespace open-api-test-inter: services "xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal" not found

    Should you encounter this error, verify that the Helm application name is short enough and then proceed with redeployment.

Deploy a new NGINX Ingress controller service

Upon creation of an ACK cluster, an NGINX Ingress controller with two pods is automatically deployed, along with an Internet-facing SLB instance as the frontend load balancing Service.

Follow these steps to deploy a completely independent NGINX Ingress controller service within the ACK cluster.

  1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

  2. On the Cluster List page, click the desired cluster name. Then, in the left-side navigation pane, select Applications > Helm.

  3. On the Helm page, click Deploy. In the Basic Information step, configure the parameters based on the following table.

    Parameter

    Example Value

    Application Name

    ack-ingress-nginx

    Note

    You can customize it, and pay attention to the Service name length restriction.

    Namespace

    kube-system

    Source

    Default is App Market

    Chart

    • Scenarios: Select All.

    • Supported Architecture: Select amd64.

    • Search Box: Search for Ack-ingress-nginx.

      • Select ack-ingress-nginx for clusters of version 1.20 and below.

      • Select ack-ingress-nginx-v1 for clusters of version 1.22 and above.

    Choose the Chart name that matches your cluster version and click Next.

  4. On the Parameter Configuration page, select Chart Version , and then click OK.

    Note

    Chart version 4.0.17 and later (ack-ingress-nginx-v1 version 1.8.0-aliyun.1 and later) are compatible only with Kubernetes 1.22 or newer. If your cluster runs Kubernetes 1.20, opt for chart version 4.0.16 (ack-ingress-nginx-v1 version 1.2.1-aliyun.1).

    The primary parameters for ack-ingress-nginx-v1 are listed below.

    Parameter

    Description

    controller.image.repository

    The image repository of ingress-nginx.

    controller.image.tag

    The image version of ingress-nginx. For more information, see Nginx Ingress Controller.

    controller.ingressClassResource.name

    Set the name of the IngressClass corresponding to the Ingress controller.

    Important

    This parameter serves as a replacement for the controller.ingressClass parameter in templates for versions below 1.22 and can be used normally in the kubernetes.io/ingress.class annotation. The IngressClass names created by different sets of Ingress controllers in the same cluster must be unique and cannot be set to the nginx keyword (nginx is the listening identifier of the default Ingress controller in the cluster).

    controller.ingressClassResource.controllerValue

    Set the Controller Class corresponding to the Ingress controller.

    Important

    The Controller Class created by different sets of Ingress controllers in the same cluster must be unique and cannot be set to the k8s.io/ingress-nginx keyword (k8s.io/ingress-nginx is the listening identifier of the default Ingress controller in the cluster).

    controller.replicaCount

    Set the number of replicas for the Ingress controller pod.

    controller.service.enabled

    Specifies whether to enable SLB access (including public and private networks).

    controller.service.external.enabled

    Specifies whether to enable Internet-facing SLB access. If you do not need to enable it, set it to false.

    controller.service.internal.enabled

    Specifies whether to enable internal-facing SLB access. If you need to enable it, set it to true.

    controller.kind

    Set the deployment pattern of the Ingress controller. Optional values: Deployment and DaemonSet.

    controller.electionID

    The ID used for leader election to update the Ingress endpoint status.

    Important

    If you deploy multiple sets of NGINX Ingress controllers in the same namespace through the App Market, you need to set the electionID to different values to avoid leader election conflicts.

    controller.metrics.enabled

    Specifies whether to enable the ingress-nginx metrics.

    controller.metrics.serviceMonitor.enabled

    Specifies whether to enable the ServiceMonitor, which provides the configuration on how metrics are collected.

    Note

    We recommend that you enable the ServiceMonitor after you enable the ingress-nginx metrics. Then, the system automatically generates the Prometheus configuration to collect metrics.

  5. View the NGINX Ingress controller service that has been deployed.

    Return to the Helm page. You will notice the new NGINX Ingress controller service has been successfully deployed.

Test network connectivity

Deploy a test application and configure it to use the newly deployed NGINX Ingress controller for service exposure.

  1. Deploy an NGINX test application.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          run: nginx
      template:
        metadata:
          labels:
            run: nginx
        spec:
          containers:
          - image: nginx
            imagePullPolicy: Always
            name: nginx
            ports:
            - containerPort: 80
              protocol: TCP
          restartPolicy: Always
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      sessionAffinity: None
      type: NodePort
  2. Configure the Ingress for Internet-facing service access.

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: nginx
      annotations:
        # Note that you need to set this to the INGRESS_CLASS you configured earlier.
        kubernetes.io/ingress.class: "<YOUR_INGRESS_CLASS>"
    spec:
      rules:
      - host: foo.bar.com
        http:
          paths:
          - path: /
            backend:
              service: 
                name: nginx
                port:
                  number: 80
            pathType: ImplementationSpecific
    Note

    You need to configure the annotation kubernetes.io/ingress.class.

    After deploying the application, you can carry out the following steps to view the IP addresses for both the Ingress resource's endpoint and the newly deployed NGINX Ingress controller service.

    • To query the IP address of the default nginx-ingress-lb Service's associated Internet-facing SLB instance, execute the command below.

       kubectl -n kube-system get svc nginx-ingress-lb

      You should see the following result:

      NAME               TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)                      AGE
      nginx-ingress-lb   LoadBalancer   172.19.XX.XX   192.0.XX.XX   80:31429/TCP,443:32553/TCP   2d
    • To query the IP address of the newly created nginx-ingress-lb Service's associated Internet-facing SLB instance, execute the command below.

      kubectl -n <YOUR_NAMESPACE> get svc nginx-ingress-lb

      You should see the following result:

      NAME               TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                      AGE
      nginx-ingress-lb   LoadBalancer   172.19.XX.XX   198.51.XX.XX   80:30969/TCP,443:31325/TCP   39m
    • To query the application's Ingress configuration, execute the command below.

      kubectl get ing

      You should see the following result:

      NAME      HOSTS         ADDRESS         PORTS     AGE
      nginx     foo.bar.com   198.51.XX.XX   80        5m

    The output above indicates that the Ingress IP address matches the IP address of the newly deployed NGINX Ingress controller.

  3. Test access to the application through both the default and new NGINX Ingress controllers.

     # Access the application through the default NGINX Ingress controller. The expected result is 404.
      curl -H "Host: foo.bar.com" http://192.0.2.0
    default backend - 404                                                                                                                                                                                        
      # Access the application through the new NGINX Ingress controller. The NGINX welcome page is expected.
      curl -H "Host: foo.bar.com" http://198.51.XX.XX
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    <style>
        body {
            width: 35em;
            margin: 0 auto;
            font-family: Tahoma, Verdana, Arial, sans-serif;
        }
    </style>
    </head>
    <body>
    <h1>Welcome to nginx!</h1>
    <p>If you see this page, the nginx web server is successfully installed and
    working. Further configuration is required.</p>
    
    <p>For online documentation and support please refer to
    <a href="http://nginx.org/">nginx.org</a>.<br/>
    Commercial support is available at
    <a href="http://nginx.com/">nginx.com</a>.</p>
    
    <p><em>Thank you for using nginx.</em></p>
    </body>
    </html>

The test results indicate that services exposed via separate NGINX Ingress controllers operate independently. This setup is ideal for scenarios in which certain services within the same cluster require exposure to the internet, while others are intended solely for internal access by services within the same VPC but external to the Kubernetes cluster.