This topic explains how to deploy multiple independent NGINX Ingress controllers in an Alibaba Cloud Container Service for Kubernetes (ACK) cluster, enabling differentiated service access.
Background information
You can configure both public and private network types for NGINX Ingress controllers to modify the default configuration of the NGINX Ingress controller in an ACK cluster for use with an internal-facing SLB instance. The document describes two modes that cover most use cases. However, in certain situations, such as when cluster-based public network services need to be exposed via a public network Ingress for external access, while private network services should only be accessible within the same VPC and not from the public network, you can set up two separate NGINX Ingress controller services. Each service would have a frontend connected to a different network type of SLB instance, fulfilling the requirements of these specific scenarios.
Notes
Below are notes on configuring multiple Ingress controllers, including setting a unique IngressClass and adhering to Service name length restrictions.
-
Set up a unique IngressClass
When operating multiple Ingress controllers, it's essential to assign a unique IngressClass to each to ensure smooth functioning and prevent conflicts. The parameters to be modified are as follows:
-
controller.ingressClassResource.name: Assign a unique name to each Ingress controller.
-
controller.ingressClassResource.controllerValue: Define a unique identifier for each Ingress controller.
-
-
Restrictions on Service Name Length
When you deploy the
ack-ingress-nginx-v1
Helm application and useack-ingress-nginx-v1
as the application name, the resulting Service will be named<application name>-ack-ingress-nginx-v1-controller
. For aninternal
type LoadBalancer Service, the name will be<application name>-ack-ingress-nginx-v1-controller-internal
. Make sure these names do not exceed 63 characters to avoid resource creation failures and potential error messages.no service with name xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal found in namespace open-api-test-inter: services "xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal" not found
Should you encounter this error, verify that the Helm application name is short enough and then proceed with redeployment.
Deploy a new NGINX Ingress controller service
Upon creation of an ACK cluster, an NGINX Ingress controller with two pods is automatically deployed, along with an Internet-facing SLB instance as the frontend load balancing Service.
Follow these steps to deploy a completely independent NGINX Ingress controller service within the ACK cluster.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
-
On the Cluster List page, click the desired cluster name. Then, in the left-side navigation pane, select .
-
On the Helm page, click Deploy. In the Basic Information step, configure the parameters based on the following table.
Parameter
Example Value
Application Name
ack-ingress-nginx
NoteYou can customize it, and pay attention to the Service name length restriction.
Namespace
kube-system
Source
Default is App Market
Chart
Scenarios: Select All.
Supported Architecture: Select amd64.
Search Box: Search for Ack-ingress-nginx.
Select ack-ingress-nginx for clusters of version 1.20 and below.
Select ack-ingress-nginx-v1 for clusters of version 1.22 and above.
Choose the Chart name that matches your cluster version and click Next.
-
On the Parameter Configuration page, select Chart Version , and then click OK.
NoteChart version 4.0.17 and later (ack-ingress-nginx-v1 version 1.8.0-aliyun.1 and later) are compatible only with Kubernetes 1.22 or newer. If your cluster runs Kubernetes 1.20, opt for chart version 4.0.16 (ack-ingress-nginx-v1 version 1.2.1-aliyun.1).
The primary parameters for ack-ingress-nginx-v1 are listed below.
Parameter
Description
controller.image.repository
The image repository of ingress-nginx.
controller.image.tag
The image version of ingress-nginx. For more information, see Nginx Ingress Controller.
controller.ingressClassResource.name
Set the name of the IngressClass corresponding to the Ingress controller.
ImportantThis parameter serves as a replacement for the controller.ingressClass parameter in templates for versions below 1.22 and can be used normally in the kubernetes.io/ingress.class annotation. The IngressClass names created by different sets of Ingress controllers in the same cluster must be unique and cannot be set to the nginx keyword (nginx is the listening identifier of the default Ingress controller in the cluster).
controller.ingressClassResource.controllerValue
Set the Controller Class corresponding to the Ingress controller.
ImportantThe Controller Class created by different sets of Ingress controllers in the same cluster must be unique and cannot be set to the k8s.io/ingress-nginx keyword (k8s.io/ingress-nginx is the listening identifier of the default Ingress controller in the cluster).
controller.replicaCount
Set the number of replicas for the Ingress controller pod.
controller.service.enabled
Specifies whether to enable SLB access (including public and private networks).
controller.service.external.enabled
Specifies whether to enable Internet-facing SLB access. If you do not need to enable it, set it to false.
controller.service.internal.enabled
Specifies whether to enable internal-facing SLB access. If you need to enable it, set it to true.
controller.kind
Set the deployment pattern of the Ingress controller. Optional values: Deployment and DaemonSet.
controller.electionID
The ID used for leader election to update the Ingress endpoint status.
ImportantIf you deploy multiple sets of NGINX Ingress controllers in the same namespace through the App Market, you need to set the electionID to different values to avoid leader election conflicts.
controller.metrics.enabled
Specifies whether to enable the ingress-nginx metrics.
controller.metrics.serviceMonitor.enabled
Specifies whether to enable the ServiceMonitor, which provides the configuration on how metrics are collected.
NoteWe recommend that you enable the ServiceMonitor after you enable the ingress-nginx metrics. Then, the system automatically generates the Prometheus configuration to collect metrics.
-
View the NGINX Ingress controller service that has been deployed.
Return to the Helm page. You will notice the new NGINX Ingress controller service has been successfully deployed.
Test network connectivity
Deploy a test application and configure it to use the newly deployed NGINX Ingress controller for service exposure.
-
Deploy an NGINX test application.
apiVersion: apps/v1 kind: Deployment metadata: name: nginx spec: replicas: 1 selector: matchLabels: run: nginx template: metadata: labels: run: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx ports: - containerPort: 80 protocol: TCP restartPolicy: Always --- apiVersion: v1 kind: Service metadata: name: nginx spec: ports: - port: 80 protocol: TCP targetPort: 80 selector: run: nginx sessionAffinity: None type: NodePort
-
Configure the Ingress for Internet-facing service access.
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: nginx annotations: # Note that you need to set this to the INGRESS_CLASS you configured earlier. kubernetes.io/ingress.class: "<YOUR_INGRESS_CLASS>" spec: rules: - host: foo.bar.com http: paths: - path: / backend: service: name: nginx port: number: 80 pathType: ImplementationSpecific
NoteYou need to configure the annotation
kubernetes.io/ingress.class
.After deploying the application, you can carry out the following steps to view the IP addresses for both the Ingress resource's endpoint and the newly deployed NGINX Ingress controller service.
-
To query the IP address of the default nginx-ingress-lb Service's associated Internet-facing SLB instance, execute the command below.
kubectl -n kube-system get svc nginx-ingress-lb
You should see the following result:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-ingress-lb LoadBalancer 172.19.XX.XX 192.0.XX.XX 80:31429/TCP,443:32553/TCP 2d
-
To query the IP address of the newly created nginx-ingress-lb Service's associated Internet-facing SLB instance, execute the command below.
kubectl -n <YOUR_NAMESPACE> get svc nginx-ingress-lb
You should see the following result:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-ingress-lb LoadBalancer 172.19.XX.XX 198.51.XX.XX 80:30969/TCP,443:31325/TCP 39m
-
To query the application's Ingress configuration, execute the command below.
kubectl get ing
You should see the following result:
NAME HOSTS ADDRESS PORTS AGE nginx foo.bar.com 198.51.XX.XX 80 5m
The output above indicates that the Ingress IP address matches the IP address of the newly deployed NGINX Ingress controller.
-
-
Test access to the application through both the default and new NGINX Ingress controllers.
# Access the application through the default NGINX Ingress controller. The expected result is 404. curl -H "Host: foo.bar.com" http://192.0.2.0 default backend - 404 # Access the application through the new NGINX Ingress controller. The NGINX welcome page is expected. curl -H "Host: foo.bar.com" http://198.51.XX.XX <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
The test results indicate that services exposed via separate NGINX Ingress controllers operate independently. This setup is ideal for scenarios in which certain services within the same cluster require exposure to the internet, while others are intended solely for internal access by services within the same VPC but external to the Kubernetes cluster.