Managed Service for Prometheus:Why does the ServiceMonitor or PodMonitor that I configured not take effect?

Check whether the target is detected.

1. Log on to the Managed Service for Prometheus console.

2. In the left-side navigation pane, click Instances.

3. Click the name of the Prometheus instance you want to manage. In the left-side navigation pane, click Service Discovery. On the Service Discovery page, click the Targets tab and check whether the target is detected.

   • If the target is not detected, perform the following steps:

     1. Check whether ServiceMonitor or PodMonitor is enabled.

        On the Service Discovery page, click the Configurations tab and then click the ServiceMonitor or PodMonitor tab. A status in the following figure (Icon 1) indicates that ServiceMonitor or PodMonitor is enabled. Proceed to the next step.

     2. Check whether the ServiceMonitor or PodMonitor resource that you checked in the previous step is delivered to the Container Service for Kubernetes (ACK) cluster.

        1. Log on to the ACK console.

        2. In the left-side navigation pane, click Clusters. On the Clusters page, find the cluster that you want to check and click the name of the cluster.

        3. In the left-side navigation pane, choose Workloads > Custom Resources.

        4. Click the CRDs tab and click ServiceMonitor or PodMonitor in the Name column, as indicated by Icon 1 in the following figure. You are redirected to the Resource Objects tab. Select the namespace to check whether ServiceMonitor or PodMonitor that you enabled is in the list.

           • If ServiceMonitor or PodMonitor is not in the list, you can log on to the Managed Service for Prometheus console and enable ServiceMonitor or PodMonitor again in the Actions column on the Configurations tab of the Service Discovery page.

           • If ServiceMonitor or PodMonitor is displayed, proceed to the next step.

     3. Check whether you can select the pod or service as expected by using the label selector.

        Run the following command. If no result is returned, you must configure the label selector or namespace selector to select the pod or service as expected.

        # Assume that "any" is set to "true" in the namespace selector. 
        kubectl get pod/service -l label_key=label_value -A
        # If a namespace is specified, the specified namespace is used. 
        kubectl get pod/service -l label_key=label_value -n xx_namespace_xx

     4. Check the port settings.

        1. If the port field is used, set the field to the value of spec.containers[i].ports[i].name in the pod configuration or to the value of spec.ports[i].name in the services configuration.

        2. If the targetPort field is used, set the field to the value of spec.containers[i].ports[i].containerPort in the pod configuration or to the value of spec.ports[i].targetPort in the services configuration.

           The following figure shows an example.

           

   • If the target is detected but is displayed in red, perform the following step:

     Troubleshoot the issue based on the information that is displayed in the Error column. A detected target displayed in red indicates that the Prometheus instance failed to capture monitoring data due to a target error.

   • If the target is detected and its status is UP (Icon 1) but no monitoring data is collected as expected, perform the following step:

     Click the endpoint of the target, as indicated by Icon 2 in the following preceding figure. The system directly pulls raw data from the target. You can check whether the raw data contains the monitoring metrics that you want to monitor. If the raw data contains the monitoring metrics that you want to monitor, you can click Configure Discarded Metrics in the upper-right corner on the Metrics tab of the Service Discovery page. In the dialog box that appears, check whether the discarded metrics contain the metrics that you want to monitor. You can delete the metrics and re-enable them. For more information, see Configure metrics.