This topic describes how to view control plane component dashboards in Container Service for Kubernetes (ACK) Pro clusters. This topic also provides best practices for accessing control plane components.
Prerequisites
The control plane component monitoring feature applies to ACK Pro clusters whose version is 1.16 or later.
Application Real-Time Monitoring Service (ARMS) is activated. For more information, see Activate ARMS.
The ack-arms-prometheus component is installed. For more information, see Manage components.
View component plane component dashboards
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the cluster that you want to manage. In the left-side navigation pane, choose .
On the Prometheus Monitoring page, click the Key Component Monitoring tab.
You can view the ACK Pro API server, ACK Pro ETCD, ACK Pro Scheduler, ACK Pro Cloud Controller Manager, and ACK Pro Kube Controller Manager dashboards on this page.
NoteTo use the latest Prometheus dashboards, update ack-arms-prometheus to the latest version from the Add-ons page in the console. For more information, see Manage components.
Best practices for accessing control plane components
We recommend that you follow our best practice rules when you access control plane components in an ACK Pro cluster that contains more than 100 nodes and a large amount of Kubernetes resources. This helps improve the stability of the cluster.
Retrieve data from the API server by using Informer or Lister. This helps reduce the loads of the API server and etcd.
If you want to list all data, we recommend that you add
resourceVersion=0
to the request to retrieve data from the API server cache in case the etcd is overloaded. If you want to retrieve data from the etcd, we recommend that you add the limit option to limit the number of entries returned per page.Use
Protobuf
as the API serialization protocol, which requires less memory resources and data transfer compared with JSON. For more information, see Alternate representations of resources. The following code block shows an example:kubeConfig, err := clientcmd.BuildConfigFromFlags(s.Master, s.Kubeconfig) if err != nil { return nil, err } kubeConfig.AcceptContentTypes = strings.Join([]string{runtime.ContentTypeProtobuf, runtime.ContentTypeJSON}, ",") kubeConfig.ContentType = runtime.ContentTypeProtobuf client, err := clientset.NewForConfig(restclient.AddUserAgent(kubeConfig, "content-type-example")) ...
Delete idle Kubernetes resources, such as ConfigMaps, Secrets, and persistent volume claims (PVCs), at the earliest opportunity. These idle resources can create pending pods. When the number of pending pods in the cluster exceeds 1000, the stability of kube-apsierver, kube-controller-manager, and kube-scheduler is affected.
Pay close attention to the resource usage of the control plane components, especially the CPU and memory utilization metrics, to prevent out-of-memory errors due to continuous high resource usage. If the resource usage remains high, we recommend that you delete invalid resources, optimize the actions of the client, and separate the workloads in the cluster.
Some open source components can greatly increase the loads of the control plane. You can find optimization solutions on the official websites of these components. For example, Argo Workflows provides a solution to resolve the issue of overwhelmed Kubernetes API when Argo is busy. For more information, see Running At Massive Scale.
References
Control plane component | Dashboard | Reference introduction | Link |
kube-apiserver | ACK Pro APIServer | Describes the metrics supported by kube-apiserver, provides usage notes for the dashboards of kube-apiserver, and suggests how to troubleshoot common metric anomalies. | |
cloud-controller-manager | ACK Pro Cloud Controller Manager | Describes the metrics supported by cloud-controller-mananger, provides usage notes for the dashboards of cloud-controller-manager, and suggests how to troubleshoot common metric anomalies. | |
etcd | ACK Pro ETCD | Describes the metrics supported by etcd, provides usage notes for the dashboards of etcd, and suggests how to troubleshoot common metric anomalies. | |
kube-controller-manager | ACK Pro Kube Controller Manager | Describes the metrics supported by kube-controller-manager and provides usage notes for the dashboards of kube-controller-manager. | |
kube-scheduler | ACK Pro Scheduler | Describes the metrics supported by kube-scheduler, provides usage notes for the dashboards of kube-scheduler, and suggests how to troubleshoot common metric anomalies. | |
Custom Prometheus monitoring and alerting | Custom dashboard name | Describes how to collect the metrics from kube-apiserver, etcd, kube-scheduler, cloud-controller-manager, and kube-controller-manager in an ACK Pro cluster to a self-managed Prometheus system and provides recommended alerting configurations. |