Implement network observability by using ACK Terway and Cilium Hubble - Container Service for Kubernetes

You can deploy Cilium Hubble in a Container Service for Kubernetes (ACK) cluster that uses the Terway network plug-in to view the network traffic and network policies of containers. This allows you to monitor the network architecture and business topology. This topic describes how to use Cilium Hubble to view statistics about network traffic in a container network, such as the sources and destinations of packets.

Prerequisites

An ACK managed cluster is created.

Note

Cilium Hubble supports only the One ENI for Multi-Pod mode of Terway. This mode is based on IPVLAN. Therefore, when you create the ACK cluster, you must select Terway for the Network Plug-in parameter and IPvlan for the Terway Mode parameter. Otherwise, you cannot use Cilium Hubble.

Step 1: Modify the Terway ConfigMap eni-config

Use the ACK console

Log on to the ACK console.
In the left-side navigation pane of the ACK console, click Clusters.
On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.

Modify the Terway ConfigMap eni-config.

In the left-side navigation pane of the details page, choose Configurations > ConfigMaps.
In the upper part of the ConfigMap page, select kube-system from the Namespace drop-down list, find the eni-config ConfigMap, and then click Edit YAML in the Actions column.

In the View in YAML panel, find 10-terway.conf, add the following parameters to 10-terway.conf, and then click OK.

"cilium_enable_hubble":"true",
"cilium_hubble_listen_address":":4244",
"cilium_hubble_metrics_server":":9091",
"cilium_hubble_metrics":"drop,tcp,flow,port-distribution,icmp",

Parameter	Description	Remarks
eniip_virtual_type	Specifies whether to enable the IPVLAN mode.	If the ConfigMap does not contain this parameter or the value is not set to `IPVLAN`, your cluster does not support Cilium Hubble.
cilium_enable_hubble	Specifies whether to enable Cilium Hubble to analyze network traffic.	In this example, this parameter is set to `"true"`.
cilium_hubble_listen_address	The port that is used to expose the Cilium Hubble Service.	In this example, this parameter is set to `":4244"`.
cilium_hubble_metrics_server	The port that is used to expose the Cilium Hubble metrics server.	In this example, this parameter is set to `":9091"`.
cilium_hubble_metrics	The metrics that you want to collect by using Cilium Hubble. Separate multiple metrics with commas (,).	Layer 7 network capabilities such as HTTP and DNS are not supported. Cilium Hubble can collect the following metrics: `"drop,tcp,flow,port-distribution,icmp"`. Note If you specify an excessive number of metrics, the performance of Cilium Hubble may be affected.

Restart the Terway pods for the modified ConfigMap to take effect.
1. In the left-side navigation pane of the details page, choose Workloads > Pods.
2. In the upper part of the Pods page, select kube-system from the Namespace drop-down list, enter terway-eniip in the search box, and then choose More > Delete in the Actions column of each pod whose name is terway-eniip-xxx.
3. In the message that appears, click OK.
  On the Pods page, if the Status column of terway-eniip-xxx displays Running, the pod is restarted.
4. Repeat the preceding steps to delete all Terway pods.

Use kubectl

Modify the Terway ConfigMap eni-config.

Run the following command to modify the Terway ConfigMap eni-config:
```
kubectl -n kube-system edit configmap eni-config
```

Add the following content to 10-terway.conf of the Terway ConfigMap and save the ConfigMap:

"cilium_enable_hubble":"true",
"cilium_hubble_listen_address":":4244",
"cilium_hubble_metrics_server":":9091",
"cilium_hubble_metrics":"drop,tcp,flow,port-distribution,icmp",

Parameter	Description	Remarks
eniip_virtual_type	Specifies whether to enable the IPVLAN mode.	If the ConfigMap does not contain this parameter or the value is not set to `IPVLAN`, your cluster does not support Cilium Hubble.
cilium_enable_hubble	Specifies whether to enable Cilium Hubble to analyze network traffic.	In this example, this parameter is set to `"true"`.
cilium_hubble_listen_address	The port that is used to expose the Cilium Hubble Service.	In this example, this parameter is set to `":4244"`.
cilium_hubble_metrics_server	The port that is used to expose the Cilium Hubble metrics server.	In this example, this parameter is set to `":9091"`.
cilium_hubble_metrics	The metrics that you want to collect by using Cilium Hubble. Separate multiple metrics with commas (,).	Layer 7 network capabilities such as HTTP and DNS are not supported. Cilium Hubble can collect the following metrics: `"drop,tcp,flow,port-distribution,icmp"`. Note If you specify an excessive number of metrics, the performance of Cilium Hubble may be affected

Restart the Terway pods for the modified ConfigMap to take effect.
```
kubectl rollout restart daemonset -n kube-system terway-eniip
```

Step 2: Install Cilium Hubble

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane of the cluster details page, choose Applications > Helm.

On the Helm page, click Deploy. In the Basic Information step, configure the parameters based on the following table.

Parameter	Example
Application Name	ack-terway-hubble
Namespace	kube-system
Source	Default value: Marketplace.
Chart	Use Scenarios: Select All. Supported Architecture: Select amd64. Enter ack-terway-hubble in the search box.

Select the ack-terway-hubble card and click Next.

In the Parameters step, configure the Chart Version parameter. In the Parameters section, specify the hosts parameter of ingress.The host is used to log on to Hubble UI. Configure other parameters based on your business requirements. Click OK.

Note

You must specify the parameters that are described in the following table under ingress.

Parameter	Description
enabled	Specifies whether to use the Ingress to access Hubble UI.
annotations	The annotations of the Ingress.
path	The root path of the Ingress.
hosts	The host of the Ingress.
tls	The Transport Layer Security (TLS) settings of the Ingress.

Step 3: Use Cilium Hubble

If you specify the hosts parameter for the Ingress of Cilium Hubble, you can log on to Hubble UI by accessing the Ingress host by using port 80. When you access the Ingress host, the following information appears:

Note

If the domain name that you access is not an authoritative domain name, such as ingress.local, you must run the kubectl -n kube-system get svc nginx-ingress-lb command to query the IP address of Hubble UI. Then, modify the hosts file on your computer to map ingress.local to the IP address of Hubble UI.

In the upper part of the page, you can view the topologies of pods and Services that belong to different namespaces.
In the lower part of the page, you can view the sources, destinations, ports, and forwarding states of network traffic.
If you have configured network policies, you can view packets that are dropped due to network policy mismatching.

可观测

Cilium Hubble uses the hubble-metrics Service in the kube-system namespace to expose network flow metrics. You can specify the metrics that Cilium Hubble exposes by specifying the cilium_hubble_metrics parameter in the Terway ConfigMap eni-config. You can use open source Prometheus and Managed Service for Prometheus to collect these metrics. For more information, see Use open source Prometheus to monitor an ACK cluster and Managed Service for Prometheus.

Note

For more information about the metrics that Cilium Hubble can expose, see Exported Metrics.

Category	Metric	Label	Description
drop	hubble_drop_total	reason, protocol	The number of dropped packets.
tcp	hubble_tcp_flags_total	flag, family	The number of TCP flags.
flow	hubble_flows_processed_total	type, subtype, verdict	The number of processed network flows.
port-distribution	hubble_port_distribution_total	protocol, port	The distribution of packets on destination ports.
icmp	hubble_icmp_total	family, type	The number of Internet Control Message Protocol (ICMP) messages.