You can use network policies provided by Container Service for Kubernetes (ACK) to implement policy-based access control. If you use the Terway network plug-in and you want to control network traffic based on IP addresses or ports, you can configure network policies to control access to specific applications. This topic describes how to use network policies in an ACK cluster and the common use scenarios of the network policies.
Prerequisites
An ACK dedicated cluster is created and Terway is selected as the network plug-in. For more information, see Create an ACK managed cluster or Create an ACK dedicated cluster.
The kubeconfig file of the cluster is obtained and used to connect to the cluster by using kubectl.
Usage notes
To use network policies in the ACK console, go to the Quota Center console page to submit an application.
To use network policies through a CLI, you do not need to apply to be added to a whitelist.
You can use label selectors in network policies to select namespaces and pods. If you create a large number of network policies in a cluster, the process of matching requests against rules in network policies will be time-consuming. In addition, cluster management and troubleshooting become complicated if the cluster has a large number of network policies. We recommend that you create less than 40 network policies in a cluster.
Supported items
When you create a cluster, the implementation of the network policies depends on the version of the Terway plug-in.
Version 1.9.0 and later
Component | Network policy implementation |
terway-eniip | eBPF |
If you want to use network policies to match pods in the cluster, you can use
podSelector
ornamespaceSelector
. TheipBlock
selector can be used to match addresses outside the cluster.If the initial version of Terway is earlier than 1.9.0 when you create a cluster, the implementation of network policies remains unchanged after Terway is updated to 1.9.0 or later.
The kernel version required by Extended Berkeley Packet Filter (eBPF)-based network policies must be 4.19 or later. If the kernel versions of your nodes are earlier than 4.19, the network policies do not take effect.
The preceding description applies only to regular Elastic Container Instances that run Terway. For more information about how to enable network policies on virtual nodes, see Use network policies.
Versions earlier than 1.9.0
Component | Network policy implementation |
The terway-eniip component is installed, and DataPath V2 or IPVLAN is disabled. | iptables |
The terway-eniip component is installed, and DataPath V2 or IPVLAN is enabled. | eBPF |
If you want to use network policies to match pods in the cluster, you can use
podSelector
ornamespaceSelector
. For eBPF-based network policies, theipBlock
selector can be used to match addresses outside the cluster.If the initial version of Terway is earlier than 1.9.0 when you create a cluster, the implementation of network policies remains unchanged after Terway is updated to 1.9.0 or later.
The kernel version required for the eBPF-based network policies must be 4.19 or later. If the kernel versions of your nodes are earlier than 4.19, the network policy feature reverts to the iptables mode. Network policies may not take effect as expected.
The preceding description applies only to regular Elastic Container Instances that run Terway. For more information about how to enable network policies on virtual nodes, see Use network policies.
Enable network policies
You can enable network policies when you create a cluster or after a cluster is created.
Created clusters
When you create a cluster, select Support for NetworkPolicy to enable the network policy feature. For more information, see Create an ACK Pro cluster.
Existing clusters
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose .
Click the Networking tab, find the terway-eniip component, and click Configuration. In the dialog box that appears, select Enable NetworkPolicy and then click OK.
Examples on using network policies
Prepare an application
Perform the following steps to create an application named nginx that allows access from other pods:
Use the ACK console
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the cluster that you want to manage and choose in the left-side navigation pane.
On the Deployments tab, click Create from Image. On the Create page, create an application named nginx and expose the application by using a Service. After you complete the application configuration, click Create.
You can configure the following parameters based on the description in the following table and use default settings for other parameters. For more information, see Create a Deployment from an image.
Parameter
Description
Example
Basic Information
Name
Specify a custom application name.
nginx
Replicas
Specify the value based on your business requirements.
1
Container
Image Name
Specify the name of the container image.
nginx:latest
Advanced
Services
Click Create to the right of Services. In the Create Service dialog box, set the parameters.
Name: nginx
Type:
Server Load Balancer
Public Access
Create SLB Instance
Port Mapping:
Name: nginx
Service Port: 80
Container Port: 80
Protocol: TCP
Return to the Deployments page and click Create from Image. On the Create page, create a client application named busybox to test the accessibility of the nginx Service created in the preceding step.
You can configure the following parameters for the busybox client application based on the description in the following table and use default settings for other parameters. For more information, see Create a Deployment from an image.
Parameter
Description
Example
Basic Information
Name
Specify a custom application name.
busybox
Replicas
Specify a value based on your business requirements.
1
Container Start Parameter
Image Name
Specify the name of the container image.
busybox:latest
Container Start Parameter
None.
Select stdin and tty.
Check whether the busybox client application can access the nginx Service.
On the Deployments page, find the busybox application and click its name.
On the Pods page, select a pod named busybox-{hash value} and click Terminal in the Actions column.
In the terminal, run the
wget nginx
command to access the nginx Service.The preceding output indicates that busybox can access the nginx Service.
Use the CLI
Run the following command to create an application named nginx. Then, create a Service named nginx to expose the application.
Create an application named nginx:
kubectl run nginx --image=nginx
Expected output:
pod/nginx created
Query whether the pod of the application is started:
kubectl get pod
Expected output:
NAME READY STATUS RESTARTS AGE nginx 1/1 Running 0 45s
Create a Service named nginx:
kubectl expose pod nginx --port=80
Expected output:
service/nginx exposed
View the Service:
kubectl get service
Expected output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 172.XX.XX.1 <none> 443/TCP 30m nginx ClusterIP 172.XX.XX.48 <none> 80/TCP 12s
Run the following command to create a pod named busybox and use the pod to access the nginx Service:
kubectl run busybox --rm -ti --image=busybox /bin/sh
Expected output:
If you don't see a command prompt, try pressing enter. / # / #
Obtain the nginx Service:
If you don't see a command prompt, try pressing enter. / # / # wget nginx # Enter wget nginx.
Expected output:
Connecting to nginx (172.XX.XX.48:80) saving to 'index.html' index.html 100% |****************************************************************************************************************************************************| 612 0:00:00 ETA 'index.html' saved
Scenario 1: Allow only applications with specific labels to access a Service
Use the ACK console
To use network policies in the ACK console, submit an application. For more information, see Usage notes.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, choose .
In the upper part of the Network Policies page, select the namespace where you want to create the network policy. In this example, the default namespace is selected. In the upper-right corner of the page, click Create. In the Create panel, configure the network policy.
Parameter
Description
Example
Parameter
Specify a custom network policy name.
access-nginx
Pod Selectors
Click + Specify workload type and labels and specify pod selectors for the network policy.
NoteIf you do not set the pod selector, the network policy applies to all pods in the specified namespace.
In this example, the following settings are used:
Type: Deployment
Workload: nginx
Label: app=nginx
Source
Each network policy can include a whitelist of allowed ingress rules. Each rule allows network traffic that matches both the source and port sections.
Rule:
podSelector: selects pods in the same namespace as the network policy and allows inbound traffic from these pods.
namespaceSelector: selects pods from the specified namespaces and allows inbound traffic from these pods.
ipBlock: selects the specified CIDR blocks and allows inbound traffic from these CIDR blocks.
Port: TCP and UDP are supported.
NoteIf you do not add rules, no pods are allowed to access the pods to which the network policy applies.
If DataPath V2 or IPVLAN is enabled for the cluster, you need to use the podSelector selector instead of the ipBlock selector to limit the pods that are allowed to access the pods to which the network policy applies.
In this example, no ingress rules are added.
Destination
Each network policy can include a whitelist of allowed egress rules. Each rule allows network traffic that matches both the destination and port sections.
Rule:
podSelector: selects pods in the same namespace as the network policy and allows outbound traffic to these pods.
namespaceSelector: selects pods from the specified namespaces and allows outbound traffic to these pods.
ipBlock: selects the specified CIDR blocks and allows outbound traffic to these CIDR blocks.
Port: TCP and UDP are supported.
NoteIf DataPath V2 or IPVLAN is enabled for the cluster, you need to use the podSelector selector instead of the ipBlock selector to limit the pods that can be accessed to access the pods to which the network policy applies.
In this example, no egress rules are added.
Click Next. Then, click OK.
Run the
wget nginx
command on the terminal of the busybox pod to test the accessibility of the nginx Service. For more information, see Step 5.The connection timed out because the network policy does not allow access from the busybox pod.
Modify the network policy to allow access from the busybox pod.
On the NetworkPolicies page, find the access-nginx network policy and click Edit in the Actions column.
Add an ingress rule.
Click + Add to the right of Source, and then perform the following steps:
Click + Add to the right of Rule. The following table describes the rule parameters.
Parameter
Example
Selector
podSelector
Type
Deployment
Workload
busybox
Labels
app=busybox
Click + Add to the right of Port. The following table describes the port parameters.
Parameter
Example
Protocol
TCP
Port
80
Click Next, and then click OK.
Run the
wget -O /dev/null nginx
command to test the accessibility of the nginx Service from the busybox pod after the network policy is modified.After the ingress rule is added to the network policy, the busybox pod can access the nginx Service.
Use the CLI
Run the vim
policy.yaml
command to create a file named policy.yaml with the following YAML template.vim policy.yaml
The following code block shows the content of the YAML template:
kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: access-nginx spec: podSelector: matchLabels: run: nginx ingress: - from: - podSelector: matchLabels: access: "true"
Run the following command to create a network policy by using the preceding policy.yaml file:
kubectl apply -f policy.yaml
Expected output:
networkpolicy.networking.k8s.io/access-nginx created
Run the following command to access the nginx Service. A connection timeout error is returned because the pod does not have the specified label:
kubectl run busybox --rm -ti --image=busybox /bin/sh
Test the accessibility of the nginx Service:
wget nginx
Expected output:
Connecting to nginx (172.19.XX.XX:80) wget: can't connect to remote host (172.19.XX.XX): Connection timed out
Run the following command to add the specified label to the pod:
kubectl run busybox --rm -ti --labels="access=true" --image=busybox /bin/sh
Test the accessibility of the nginx Service:
wget nginx
Expected output:
Connecting to nginx (172.21.XX.XX:80) saving to 'index.html' index.html 100% |****************************************************************************************************************************************************| 612 0:00:00 ETA 'index.html' saved
The output shows that the progress of the connection is 100%. This indicates that the pod can access the nginx Service as expected.
Scenario 2: Allow only specific source IP addresses to access a Service through an Internet-facing SLB instance
Use the ACK console
Create a network policy for the nginx Service. For more information, see Allow only applications with specific labels to access a Service.
On the Services page, find the nginx Service and copy its IP address (47.xxx.xx.x) in the External Endpoint column. Access the IP address in your browser.
The nginx Service cannot be accessed due to the preconfigured network policy.
Modify the network policy to allow specific IP addresses to access the Service.
Visit myip.ipip.net in your browser to obtain the IP address of your on-premises machine.
On the NetworkPolicies page, find the access-nginx network policy and click Edit in the Actions column. Add an ingress rule in the Edit panel.
Click + Add to the right of Source, and then perform the following steps:
Click + Add to the right of Rule. Add the IP address of your on-premises machine to the rule.
Parameter
Example
Selector
ipBlock
cidr
<IP address of your on-premises machine>/32
Example: 42.xx.xx.xx/32.
Click + Add to the right of Rule. Add the CIDR block used for Server Load Balancer (SLB) health checks to the rule.
Parameter
Example
Selector
ipBlock
cidr
100.0.0.0/8
Click + Add to the right of Port. The following table describes the port parameters.
Parameter
Example
Protocol
TCP
Port
80
Click Next, and then click OK.
On the Services page, find the nginx Service and click the IP address (47.xxx.xx.x:80) in the External Endpoint column.
After the network policy is modified, the nginx Service can be accessed through an Internet-facing SLB instance from an on-premises machine.
Use the CLI
Run the following command to create an SLB instance for the preceding nginx application. Set the Service type to
LoadBalancer
to make the application accessible over the Internet.vim nginx-service.yaml
In this example, a file named nginx-service.yaml is used.
# Copy the following YAML content to the nginx-service.yaml file: apiVersion: v1 kind: Service metadata: labels: run: nginx name: nginx-slb spec: externalTrafficPolicy: Local ports: - port: 80 protocol: TCP targetPort: 80 selector: run: nginx type: LoadBalancer
Run the following command to create a network policy by using the nginx-service.yaml file:
kubectl apply -f nginx-service.yaml
Expected output:
service/nginx-slb created
Check whether the application is exposed by using the nginx Service:
kubectl get service nginx-slb
Expected output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-slb LoadBalancer 172.19.xx.xxx 47.110.xxx.xxx 80:32240/TCP 8m
Run the following command to connect to the IP address of the created SLB instance: 47.110.xxx.xxx. The output shows that the connection failed.
wget 47.110.xxx.xxx
Expected output:
--2018-11-21 11:46:05-- http://47.110.xx.xxx/ Connecting to 47.110.XX.XX:80... failed: Connection refused.
NoteThe connection failed due to the following reasons:
The nginx Service can be accessed only by applications that have the
access=true
label.When you connect to the IP address of the SLB instance from your on-premises machine, you are accessing the nginx Service from outside the Kubernetes cluster. This is not the same as when you use a network policy to allow only applications with specified labels to access the nginx Service.
Solution: Modify the network policy to allow the source IP address to access the nginx Service.
Run the following command to query the IP address of your on-premises machine:
curl myip.ipip.net
Expected output:
IP address: 10.0.x.x. From: China Beijing Beijing # This is an example. Use the actual IP address.
Run the following command to modify the policy.yaml file:
vim policy.yaml
Example of the policy.yaml file:
# The following content is an example of the YAML file. kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: access-nginx spec: podSelector: matchLabels: run: nginx ingress: - from: - podSelector: matchLabels: access: "true" - ipBlock: cidr: 100.64.0.0/10 - ipBlock: cidr: 10.0.0.1/24 # This is an example. Use the actual IP address.
Run the following command to create a network policy by using the preceding policy.yaml file:
kubectl apply -f policy.yaml
Expected output:
networkpolicy.networking.k8s.io/access-nginx unchanged
NoteThe request may come from different IP addresses. Therefore, specify a CIDR block with a mask length of 24.
The IP addresses used by SLB for health checks are within the
100.64.0.0/10
CIDR block. Therefore, make sure that you add100.64.0.0/10
to the rule.
Run the following command to create the resources that are provisioned for the nginx Service:
kubectl run busybox --rm -ti --labels="access=true" --image=busybox /bin/sh
Access the nginx Service:
wget 47.110.XX.XX
Expected output:
Connecting to 47.110.XX.XX (47.110.XX.XX:80) index.html 100% |***********************************************************| 612 0:00:00 ETA
If the output shows 100%, this indicates that the nginx Service can be accessed.
Scenario 3: Allow a pod to access a specific address
Use the ACK console
In this section, www.aliyun.com and registry.aliyuncs.com are used as examples. The network policy allows a pod to access only registry.aliyuncs.com.
Use the Ping command to query the IP address (120.55.XXX.XXX) to which the registry.aliyuncs.com domain name is resolved.
Create a network policy and add an egress rule to allow the busybox application to access only registry.aliyuncs.com.
In the upper-right corner of the NetworkPolicies page, click Create. In the Create panel, set the parameters.
Type: Deployment
Workload: busybox
Label: app=busybox
Selector: ipBlock
CIDR: 120.55.XXX.XXX/32
Click + Add to the right of Rule. The following table describes the rule that selects all namespaces.
Click + Add to the right of Port. Add UDP port 53 to ensure that the application can perform DNS lookups.
Rules:
Selector: namespaceSelector
Namespace: All
Port:
Protocol: UDP
Port: 53
Click Next, and then click OK.
Go to the terminal of the busybox pod and run the following commands to access www.aliyun.com and registry.aliyuncs.com in sequence:
nc -vz -w 1 www.aliyunc.com 443
nc -vz -w 1 registry.aliyuncs.com 443
After the network policy is added, the busybox application can access only registry.aliyuncs.com.
For more information about the parameters, see Allow only applications with specific labels to access a Service. The following table provides a sample configuration.
Parameter
Description
Example
Parameter
Specify a custom network policy name.
busybox-policy
Pod Selectors
Click + Specify workload type and labels and specify pod selectors for the network policy.
NoteIf you do not set the pod selector, the network policy applies to all pods in the specified namespace.
In this example, the following settings are used:
Destination
Click + Add to the right of Destination. Then, click + Add to the right of Rule.
Select ipBlock and add the IP address to which registry.aliyuncs.com is resolved. In this example, the IP address is 120.55.XXX.XXX/32.
Click + Add to the right of Destination. Select namespaceSelector and allow outbound traffic to pods in all namespaces through UDP port 53. This ensures that the application can perform DNS lookups.
Use the CLI
Run the following command to query the IP addresses to which www.aliyun.com is resolved:
dig +short www.aliyun.com
Expected output:
www-jp-de-intl-adns.aliyun.com. www-jp-de-intl-adns.aliyun.com.gds.alibabadns.com. v6wagbridge.aliyun.com. v6wagbridge.aliyun.com.gds.alibabadns.com. 106.XX.XX.21 140.XX.XX.4 140.XX.XX.13 140.XX.XX.3
Run the following command to create a file named busybox-policy:
vim busybox-policy.yaml
Example of the busybox-policy file:
# The following content is a sample YAML file. kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: busybox-policy spec: podSelector: matchLabels: run: busybox egress: - to: - ipBlock: cidr: 106.XX.XX.21/32 - ipBlock: cidr: 140.XX.XX.4/32 - ipBlock: cidr: 140.XX.XX.13/32 - ipBlock: cidr: 140.XX.XX.3/32 - to: - ipBlock: cidr: 0.0.0.0/0 - namespaceSelector: {} ports: - protocol: UDP port: 53
NoteEgress rules are configured in the busybox-policy file to limit outbound access from the application. You must allow the application to access UDP port 53. Otherwise, Domain Name System (DNS) resolution does not work.
Run the following command to create a network policy based on the busybox-policy file:
kubectl apply -f busybox-policy.yaml
Expected output:
networkpolicy.networking.k8s.io/busybox-policy created
Run the following command to create the busybox application:
kubectl run busybox --rm -ti --image=busybox /bin/sh
Access a website other than www.aliyun.com, such as www.taobao.com:
wget www.taobao.com
Expected output:
Connecting to www.taobao.com (64.13.XX.XX:80) wget: can't connect to remote host (64.13.XX.XX): Connection timed out
The output shows the can't connect to remote host message. This indicates that the application failed to access the domain name.
Run the following command to access www.aliyun.com:
wget www.aliyun.com
Expected output:
Connecting to www.aliyun.com (140.205.XX.XX:80) Connecting to www.aliyun.com (140.205.XX.XX:443) wget: note: TLS certificate validation not implemented index.html 100% |***********************************************************| 462k 0:00:00 ETA
The output shows that the progress of the connection is 100%. This indicates that the application can access the domain name as expected.
Scenario 4: Control pod access to the Internet in specific namespaces
The operations in this section may affect services that communicate with the Internet. We recommend that you create an empty namespace to perform the following operations.
Use the ACK console
In the upper-right corner of the NetworkPolicies page, click Create. In the Create panel, set the parameters.
For more information about the parameters, see Allow only applications with specific labels to access a Service. The following table provides a sample configuration.
Parameter
Example
Parameter
deny-public-net
Pod Selectors
Set Type to All.
Source
Add the following ingress rules:
Set namespaceSelector to All.
Set ipBlock to 0.0.0.0/0.
Destination
Add egress rules that allow the pod to access only the internal network.
Set namespaceSelector to All. This allows the pod to access all other pods in the internal network.
Create three rules and set ipBlock to the following CIDR blocks in sequence:
10.0.0.0/8
172.16.0.0/12
192.168.0.0/16
NoteYou cannot specify multiple CIDR blocks for the ipBlock parameter in one rule.
Click Next, and then click OK.
On the Basic Information tab of the cluster details page, obtain the private IP address and port.
Run the following command on the terminal of the busybox pod to test whether the pod can access the Internet and the internal network:
nc -vz -w 1 www.aliyunc.com 443
nc -vz -w 1 10.xx.xx.xx:<IP port> # Specify the private IP address.
The output shows that the pod can access the internal network but not the Internet.
Use the CLI
Run the following command to create a namespace for testing.
Create a namespace named test-np.
kubectl create ns test-np
Expected output:
namespace/test-np created
Run the following command to create a default network policy that allows pods in the test-np namespace to access only internal services:
vim default-deny.yaml
Example of the default-deny.yaml file:
# The following content is a sample YAML file. kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: namespace: test-np name: deny-public-net spec: podSelector: {} ingress: - from: - ipBlock: cidr: 0.0.0.0/0 egress: - to: - ipBlock: cidr: 192.168.0.0/16 - ipBlock: cidr: 172.16.0.0/12 - ipBlock: cidr: 10.0.0.0/8
Check whether a network policy is created by using the default-deny.yaml file.
kubectl apply -f default-deny.yaml
Expected output:
networkpolicy.networking.k8s.io/deny-public-net created
View the network policy:
kubectl get networkpolicy -n test-np
Expected output:
NAME POD-SELECTOR AGE deny-public-net <none> 1m
Run the following command to allow pods with specific labels to access the Internet.
vim allow-specify-label.yaml
In this example, the
public-network=true
label is used.# The following content is a sample YAML file. kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: allow-public-network-for-labels namespace: test-np spec: podSelector: matchLabels: public-network: "true" ingress: - from: - ipBlock: cidr: 0.0.0.0/0 egress: - to: - ipBlock: cidr: 0.0.0.0/0 - namespaceSelector: matchLabels: ns: kube-system
Run the following command to create the network policy:
kubectl apply -f allow-specify-label.yaml
Expected output:
networkpolicy.networking.k8s.io/allow-public-network-for-labels created
View the network policy:
kubectl get networkpolicy -n test-np
Expected output:
NAME POD-SELECTOR AGE allow-public-network-for-labels public-network=true 1m deny-public-net <none> 3m
Run the following command to verify that a pod without the specified label cannot access the Internet:
kubectl run -it --namespace test-np --rm --image busybox busybox-intranet
ping aliyun.com
Expected output:
PING aliyun.com (106.11.2xx.xxx): 56 data bytes ^C --- aliyun.com ping statistics --- 9 packets transmitted, 0 packets received, 100% packet loss
The output shows 0 packets received. This indicates that the pod failed to access the Internet.
NoteThe pod failed to access the Internet because the deny-public-net network policy does not allow pods in the test-np namespace to access the Internet. Therefore, pods with default labels cannot access the Internet.
Run the following command to verify that a pod with the public-network=true label can access the Internet:
kubectl run -it --namespace test-np --labels public-network=true --rm --image busybox busybox-internet
ping aliyun.com
Expected output:
PING aliyun.com (106.11.1xx.xx): 56 data bytes 64 bytes from 106.11.1xx.xx: seq=0 ttl=47 time=4.235 ms 64 bytes from 106.11.1xx.xx: seq=1 ttl=47 time=4.200 ms 64 bytes from 106.11.1xx.xx: seq=2 ttl=47 time=4.182 ms ^C --- aliyun.com ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 4.182/4.205/4.235 ms
The output shows 0% packet loss. This indicates that the pod can access the Internet.
NoteThe pod can access the Internet because the allow-public-network-for-labels network policy allows pods with the public-network=true label to access the Internet. The busybox-internet pod has this label and therefore can access the Internet.