This topic explains how to deploy multiple independent NGINX Ingress controllers in an Alibaba Cloud Container Service for Kubernetes (ACK) cluster, enabling differentiated service access.
Background information
You can configure both public and private network types for NGINX Ingress controllers to modify the default configuration of the NGINX Ingress controller in an ACK cluster for use with an internal-facing SLB instance. The document describes two modes that cover most use cases. However, in certain situations, such as when cluster-based public network services need to be exposed via a public network Ingress for external access, while private network services should only be accessible within the same VPC and not from the public network, you can set up two separate NGINX Ingress controller services. Each service would have a frontend connected to a different network type of SLB instance, fulfilling the requirements of these specific scenarios.
Notes
Below are notes on configuring multiple Ingress controllers, including setting a unique IngressClass and adhering to Service name length restrictions.
-
Set up a unique IngressClass
When operating multiple Ingress controllers, it's essential to assign a unique IngressClass to each to ensure smooth functioning and prevent conflicts. The parameters to be modified are as follows:
-
Restrictions on Service Name Length
When you deploy the ack-ingress-nginx-v1
Helm application and use ack-ingress-nginx-v1
as the application name, the resulting Service will be named <application name>-ack-ingress-nginx-v1-controller
. For an internal
type LoadBalancer Service, the name will be <application name>-ack-ingress-nginx-v1-controller-internal
. Make sure these names do not exceed 63 characters to avoid resource creation failures and potential error messages.
no service with name xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal found in namespace open-api-test-inter: services "xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal" not found
Should you encounter this error, verify that the Helm application name is short enough and then proceed with redeployment.
Deploy a new NGINX Ingress controller service
Upon creation of an ACK cluster, an NGINX Ingress controller with two pods is automatically deployed, along with an Internet-facing SLB instance as the frontend load balancing Service.
Follow these steps to deploy a completely independent NGINX Ingress controller service within the ACK cluster.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
-
On the Cluster List page, click the desired cluster name. Then, in the left-side navigation pane, select .
-
On the Helm page, click Deploy. In the Basic Information step, configure the parameters based on the following table.
Parameter | Example Value |
Application Name | ack-ingress-nginx |
Namespace | kube-system |
Source | Default is App Market |
Chart | |
Choose the Chart name that matches your cluster version and click Next.
-
On the Parameter Configuration page, select Chart Version , and then click OK.
Note
Chart version 4.0.17 and later (ack-ingress-nginx-v1 version 1.8.0-aliyun.1 and later) are compatible only with Kubernetes 1.22 or newer. If your cluster runs Kubernetes 1.20, opt for chart version 4.0.16 (ack-ingress-nginx-v1 version 1.2.1-aliyun.1).
The primary parameters for ack-ingress-nginx-v1 are listed below.
Parameter | Description |
controller.image.repository | The image repository of ingress-nginx. |
controller.image.tag | The image version of ingress-nginx. For more information, see Nginx Ingress Controller. |
controller.ingressClassResource.name | Set the name of the IngressClass corresponding to the Ingress controller. Important This parameter serves as a replacement for the controller.ingressClass parameter in templates for versions below 1.22 and can be used normally in the kubernetes.io/ingress.class annotation. The IngressClass names created by different sets of Ingress controllers in the same cluster must be unique and cannot be set to the nginx keyword (nginx is the listening identifier of the default Ingress controller in the cluster). |
controller.ingressClassResource.controllerValue | Set the Controller Class corresponding to the Ingress controller. Important The Controller Class created by different sets of Ingress controllers in the same cluster must be unique and cannot be set to the k8s.io/ingress-nginx keyword (k8s.io/ingress-nginx is the listening identifier of the default Ingress controller in the cluster). |
controller.replicaCount | Set the number of replicas for the Ingress controller pod. |
controller.service.enabled | Specifies whether to enable SLB access (including public and private networks). |
controller.service.external.enabled | Specifies whether to enable Internet-facing SLB access. If you do not need to enable it, set it to false. |
controller.service.internal.enabled | Specifies whether to enable internal-facing SLB access. If you need to enable it, set it to true. |
controller.kind | Set the deployment pattern of the Ingress controller. Optional values: Deployment and DaemonSet. |
controller.electionID | The ID used for leader election to update the Ingress endpoint status. Important If you deploy multiple sets of NGINX Ingress controllers in the same namespace through the App Market, you need to set the electionID to different values to avoid leader election conflicts. |
controller.metrics.enabled | Specifies whether to enable the ingress-nginx metrics. |
controller.metrics.serviceMonitor.enabled | Specifies whether to enable the ServiceMonitor, which provides the configuration on how metrics are collected. Note We recommend that you enable the ServiceMonitor after you enable the ingress-nginx metrics. Then, the system automatically generates the Prometheus configuration to collect metrics. |
-
View the NGINX Ingress controller service that has been deployed.
Return to the Helm page. You will notice the new NGINX Ingress controller service has been successfully deployed.
Test network connectivity
Deploy a test application and configure it to use the newly deployed NGINX Ingress controller for service exposure.
-
Deploy an NGINX test application.
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
run: nginx
template:
metadata:
labels:
run: nginx
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
ports:
- containerPort: 80
protocol: TCP
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
sessionAffinity: None
type: NodePort
-
Configure the Ingress for Internet-facing service access.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx
annotations:
kubernetes.io/ingress.class: "<YOUR_INGRESS_CLASS>"
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /
backend:
service:
name: nginx
port:
number: 80
pathType: ImplementationSpecific
Note
You need to configure the annotation kubernetes.io/ingress.class
.
After deploying the application, you can carry out the following steps to view the IP addresses for both the Ingress resource's endpoint and the newly deployed NGINX Ingress controller service.
-
To query the IP address of the default nginx-ingress-lb Service's associated Internet-facing SLB instance, execute the command below.
kubectl -n kube-system get svc nginx-ingress-lb
You should see the following result:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-ingress-lb LoadBalancer 172.19.XX.XX 192.0.XX.XX 80:31429/TCP,443:32553/TCP 2d
-
To query the IP address of the newly created nginx-ingress-lb Service's associated Internet-facing SLB instance, execute the command below.
kubectl -n <YOUR_NAMESPACE> get svc nginx-ingress-lb
You should see the following result:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-ingress-lb LoadBalancer 172.19.XX.XX 198.51.XX.XX 80:30969/TCP,443:31325/TCP 39m
-
To query the application's Ingress configuration, execute the command below.
You should see the following result:
NAME HOSTS ADDRESS PORTS AGE
nginx foo.bar.com 198.51.XX.XX 80 5m
The output above indicates that the Ingress IP address matches the IP address of the newly deployed NGINX Ingress controller.
-
Test access to the application through both the default and new NGINX Ingress controllers.
curl -H "Host: foo.bar.com" http://192.0.2.0
default backend - 404
curl -H "Host: foo.bar.com" http://198.51.XX.XX
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
The test results indicate that services exposed via separate NGINX Ingress controllers operate independently. This setup is ideal for scenarios in which certain services within the same cluster require exposure to the internet, while others are intended solely for internal access by services within the same VPC but external to the Kubernetes cluster.