How to use Application Monitoring eBPF Edition to monitor cluster topologies - Container Service for Kubernetes

Alibaba Cloud Application Monitoring eBPF Edition is a collection of observability services that are developed based on the Extended Berkeley Packet Filter (eBPF) technology for Kubernetes clusters. After you install the Application Monitoring eBPF Edition component in the cluster, you can monitor Kubernetes clusters based on metrics, traces, logs, and events.

Prerequisites

A Container Service for Kubernetes (ACK) cluster is created. For more information, see Create an ACK managed cluster.
Application Real-Time Monitoring Service (ARMS) is activated. For more information, see Activate ARMS.
Ensure that the current environment supports Application Monitoring eBPF Edition. For more information, see Requirements and limitations on operating systems of Application Monitoring eBPF Edition.

Background information

Kubernetes workloads run in resource pools that consist of nodes. Therefore, the traces of pods are difficult to identify and the topology is complex. The greatest challenge is how to monitor the workloads in an ACK cluster in a visualized manner and how to visualize the throughput of the ACK cluster. Application Monitoring eBPF Edition uses the eBPF technology to obtain the Rate, Errors, and Duration (RED) data of containers without code intrusion. It can efficiently identify performance issues in containers and pods. In addition, Application Monitoring eBPF Edition can identify the Services and controller workloads, such as Deployments, StatefulSets, and DaemonSets, that are related to the issues. This improves troubleshooting efficiency. For more information, see What is Application Monitoring eBPF Edition.

Connect to an ACK cluster

To connect an ACK cluster to Application Monitoring eBPF Edition, install the following monitoring agents in the cluster:

Prometheus monitoring agent: ack-arms-prometheus
Note
The metrics that are used in Application Monitoring eBPF Edition are collected by Managed Service for Prometheus. Therefore, you must install the Managed Service for Prometheus agent.
Application Monitoring eBPF Edition component: ack-arms-cmonitor
Before you install ack-arms-cmonitor in a cluster, you must check whether ARMS Addon Token exists in the cluster. If ARMS Addon Token exists, ARMS performs password-free authorization. After the agent is installed, you can use Application Monitoring eBPF Edition. If ARMS Addon Token does not exist, an error occurs due to insufficient permissions. You must attach the policies that provide full permissions on ARMS and Tracing Analysis to the worker Resource Access Management (RAM) role.
The following section describes how to check whether ARMS Addon Token exists and how to attach the policies to the worker RAM role:
Check whether the cluster has an ARMS Addon Token
1. Log on to the ACK console. In the left-side navigation pane, click Clusters.
2. On the Clusters page, click the name of the cluster that you want to manage and choose Configurations > Secrets in the left-side navigation pane.
3. In the upper part of the Secret page, select kube-system from the Namespace drop-down list and check whether addon.arms.token exists.
Attach the policies that provide full permissions on ARMS and Tracing Analysis to the worker RAM role
1. Log on to the ACK console. In the left-side navigation pane, click Clusters.
2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, click Cluster Information.
3. Click Basic Information on the Cluster Information page. On the page that appears, click the hyperlink on the right side of Worker RAM Role.
4. On the Permission tab, click Grant Permission.
5. In the Policy section of the Grant Permission panel, enter the keywords of the following policies in the search box. Click the policies to add them to the right-side Selected list, and complete the grant permissions step as prompted.
  - AliyunTracingAnalysisFullAccess: provides full permissions on Tracing Analysis.
  - AliyunARMSFullAccess: provides full permissions on ARMS.
Note
- ACK managed cluster: ARMS Addon Token may not exist in specific ACK managed clusters. If you use an ACK managed cluster, we recommend that you first check whether ARMS Addon Token exists. If ARMS Addon Token does not exist, you must manually complete authorization.
- ACK dedicated cluster: By default, ACK dedicated clusters do not have ARMS Addon Token. You must manually complete authorization.
- Registered cluster: By default, registered clusters do not have ARMS Addon Token. You must manually complete authorization. Registered clusters do not have a worker RAM role. You cannot manually attach the ARMS and Tracing Analysis permission policies to the RAM role. For more information about how to install the Application Monitoring eBPF Edition component ack-arms-cmonitor in a registered cluster, see Install a Kubernetes Monitoring agent for a registered cluster.

Integrate applications to Application Monitoring eBPF Edition

Integrate manually

Install the component for existing applications

If the ack-arms-cmonitor agent has been installed in your application, check whether the version is 4.0.0 or later. Log on to the ACK console, choose Operations > Add-ons in the left-side navigation pane of the cluster details page. If the version is earlier than 4.0.0, click Upgrade to upgrade the agent to the latest version. To install the ack-arms-cmonitor agent, perform the following steps:

Log on to the ACK console. In the left-side navigation pane, click Clusters. On the Clusters page, click the name of the cluster.
In the left-side navigation pane, choose Operations > Cluster Topology.
On the Cluster Topology page, click Install. The ACK console automatically installs the agent.
Note
The default namespace is arms-prom.
If you are using an ACK dedicated cluster, update the AccessKey pair.
1. In the left-side navigation pane, choose Applications > Helm.
2. Find arms-cmonitor and click Update. Set the accessKey and accessKeySecret parameters to the AccessKey ID and secret of your Alibaba Cloud account. For more information, see Obtain the AccessKey pair of the primary center.
  Important
  You must make sure that the AliyunARMSFullAccess and AliyunSTSAssumeRoleAccess policies are attached to your Alibaba Cloud account.
3. Click OK.

Install the component when you create a new application

Log on to the ACK console. On the Clusters page, find the cluster and click Applications in the Actions column.
On the Deployments page, click Create from YAML in the upper-right corner of the page.

Select a template from the Sample Template drop-down list and add the following labels to the spec.template.metadata directory in the YAML template.

labels:
  armseBPFAutoEnable: "on"
  armseBPFCreateAppName: "<your-deployment-name>"    //Replace <your-deployment-name> with the application name.

The following YAML template shows how to create a Deployment and enable Application Monitoring eBPF Edition:

View YAML template

apiVersion: v1
kind: Namespace
metadata:
  name: arms-demo
---
apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1
kind: Deployment
metadata:
  name: arms-springboot-demo
  namespace: arms-demo
  labels:
    app: arms-springboot-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: arms-springboot-demo
  template:
    metadata:
      labels:
        app: arms-springboot-demo
        armseBPFAutoEnable: "on"
        armseBPFCreateAppName: "arms-k8s-demo"
    spec:
      containers:
        - resources:
            limits:
              cpu: 0.5
          image: registry.cn-hangzhou.aliyuncs.com/arms-docker-repo/arms-springboot-demo:v0.1
          imagePullPolicy: Always
          name: arms-springboot-demo
          env:
            - name: SELF_INVOKE_SWITCH
              value: "true"
            - name: COMPONENT_HOST
              value: "arms-demo-component"
            - name: COMPONENT_PORT
              value: "6666"
            - name: MYSQL_SERVICE_HOST
              value: "arms-demo-mysql"
            - name: MYSQL_SERVICE_PORT
              value: "3306"
---
apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1
kind: Deployment
metadata:
  name: arms-springboot-demo-subcomponent
  namespace: arms-demo
  labels:
    app: arms-springboot-demo-subcomponent
spec:
  replicas: 2
  selector:
    matchLabels:
      app: arms-springboot-demo-subcomponent
  template:
    metadata:
      labels:
        app: arms-springboot-demo-subcomponent
        armseBPFAutoEnable: "on"
        armseBPFCreateAppName: "arms-k8s-demo-subcomponent"
    spec:
      containers:
        - resources:
            limits:
              cpu: 0.5
          image: registry.cn-hangzhou.aliyuncs.com/arms-docker-repo/arms-springboot-demo:v0.1
          imagePullPolicy: Always
          name: arms-springboot-demo-subcomponent
          env:
            - name: SELF_INVOKE_SWITCH
              value: "false"
            - name: MYSQL_SERVICE_HOST
              value: "arms-demo-mysql"
            - name: MYSQL_SERVICE_PORT
              value: "3306"
---
apiVersion: v1
kind: Service
metadata:
  labels:
    name: arms-demo-component
  name: arms-demo-component
  namespace: arms-demo
spec:
  ports:
    # the port that this service should serve on
    - name: arms-demo-component-svc
      port: 6666
      targetPort: 8888
  # label keys and values that must match in order to receive traffic for this service
  selector:
    app: arms-springboot-demo-subcomponent
---
apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1
kind: Deployment
metadata:
  name: arms-demo-mysql
  namespace: arms-demo
  labels:
    app: mysql
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - resources:
            limits:
              cpu: 0.5
          image: registry.cn-hangzhou.aliyuncs.com/arms-docker-repo/arms-demo-mysql:v0.1
          name: mysql
          ports:
            - containerPort: 3306
              name: mysql
---
apiVersion: v1
kind: Service
metadata:
  labels:
    name: mysql
  name: arms-demo-mysql
  namespace: arms-demo
spec:
  ports:
    # the port that this service should serve on
    - name: arms-mysql-svc
      port: 3306
      targetPort: 3306
  # label keys and values that must match in order to receive traffic for this service
  selector:
    app: mysql
--

Integrate automatically

Log on to the ARMS console.
In the left-side navigation pane, click Integration Center. In the Server Applications section, click the Application Monitoring eBPF card.
On the Access tab, select the Container Service cluster where the application resides.
Select the workload that corresponds to the application and click OK.
Note
You can select multiple applications at the same time.
Enter a name for the application and click Run.
After the components are installed, restart the application.
On the Access Management page, you can view all applications and components that are connected to ARMS.
On the Access Management page, click a cluster name to view the basic information, connected components, and metrics of the cluster. You can also explore metrics and configure ARMS agents.