Application Real-Time Monitoring Service:Use Pushgateway to push metric data

Overview

If the Prometheus server cannot pull data from your data source on a regular basis (for example, in an environment where no stable network connection is available), you can use Pushgateway to push data. After metric data is sent from your data source to Pushgateway, the Prometheus server periodically obtains the metric data. To implement this, perform the following steps:

1. Obtain the Pushgateway endpoint: Obtain the Pushgateway endpoint in the Managed Service for Prometheus console.

2. Report data: You can run the curl command or use an open source Pushgateway SDK to push metric data. This ensures that metric data can be collected and monitored by Managed Service for Prometheus in a timely and reliable manner.

3. Configure data protection (optional): The standard Pushgateway protocol does not contain content related to data protection. The Pushgateway SDK provides only basic authentication and does not provide more advanced and general authentication. This means that any client can push data once it obtains the Pushgateway endpoint. To ensure data security, you can obtain a token in the Managed Service for Prometheus console to implement the standard JSON web token (JWT) authentication protocol.

   

Prerequisites

A Prometheus instance is created. For more information, see the following topics:

• Monitor an ACK cluster

• Monitor ECS instances

• Monitor an Alibaba Cloud service

Step 1: Obtain the Pushgateway endpoint

1. Go to the details page of your Prometheus instance: Log on to the Application Real-Time Monitoring Service (ARMS) console. In the left-side navigation pane, choose Managed Service for Prometheus > Instances. Click the name of the Prometheus instance that you want to manage.

   

2. Obtain the Pushgateway endpoint: In the left-side navigation pane, click Settings. On the Settings tab, obtain the public URL in the Push Gateway URL section.

   

Step 2: Report data

completionTime := prometheus.NewGauge(prometheus.GaugeOpts{
    Name: "db_backup_last_completion_timestamp_seconds",
    Help: "The timestamp of the last successful completion of a DB backup.",
})
completionTime.SetToCurrentTime()
url :   = "https://cn-hangzhou.arms.aliyuncs.com/prometheus/52b12ea9cf4bb9e35****/16727530178****/1df8lj***/cn-hangzhou/api/v2"
pusher := push.New(url, "test").
    Collector(completionTime).Client(http.DefaultClient).
    Grouping("key1", "test1").Grouping("key2", "dfdf/sdsd/").
    Format(expfmt.FmtProtoDelim)
if err := pusher.Push(); err != nil {
    fmt.Println("Could not push completion time to PushGateway: ", err)
}

CollectorRegistry registry = new CollectorRegistry();
Gauge duration = Gauge.build()
        .name("my_batch_job_duration_seconds").help("Duration of my batch job in seconds.").register(registry);
Gauge.Timer durationTimer = duration.startTimer();
try {
    // Your code here.

    // This is only added to the registry after success,
    // so that a previous success in the Pushgateway isn't overwritten on failure.
    Gauge lastSuccess = Gauge.build()
            .name("my_batch_job_last_success").help("Last time my batch job succeeded, in unixtime.").register(registry);
    lastSuccess.setToCurrentTime();
} finally {
    durationTimer.setDuration();
    PushGateway pg = new PushGateway(new URL("https://cn-hangzhou.arms.aliyuncs.com/prometheus/52b12ea9cf4bb9e35****/16727530178****/1df8lj***/cn-hangzhou/api/v2"));
    pg.pushAdd(registry, "my_batch_job");
}

echo "some_metric 3.14" | curl -H "Content-Type: text/plain; version=0.0.4; charset=utf-8" --data-binary @- https://cn-hangzhou.arms.aliyuncs.com/prometheus/51bbea9ck41b9e35****/16727530178****/1df8lj***/cn-hangzhou/api/v2/metrics/label_key_1/label_value_1/label_key_2/label_value_2

Step 3: Configure data protection (optional)

1. Obtain a token: In the left-side navigation pane, click Settings. On the Settings tab, click Generate Token in the Token section.

   

2. Pass the token: After a token is generated, you can view the value of the token. You can use one of the following methods to pass the token:

   • Method 1: Add the token to the request header of the client request. Then, you can push metric data. Otherwise, the system rejects data writes. The following figure shows how to add the token to the request header of the client request.

   • Method 2: The Pushgateway SDK supports only basic authentication and does not support JWT authentication. If you want to use the Pushgateway SDK and implement advanced authentication at the same time, you can use the BasicAuth interface and set the Password field to Token. The service side is compatible with the authentication method. Method 1 may increase your development cost. In this example, Pushgateway SDK for Go is used.

     pusher := push.New(url, "test").
             Collector(completionTime).Client(http.DefaultClient).
             Grouping("key1", "test1").Grouping("key2", "dfdf/sdsd/").
             .BasicAuth("admin", "Token value").
             Format(expfmt.FmtProtoDelim)

Check the result

You can use Grafana to check whether metric data is pushed.

1. Go to the ApiServer dashboard: In the left-side navigation pane, click Dashboards. On the Dashboards page, click ApiServer.

2. View data on the Explore page: In the left-side navigation pane, move the pointer over the icon (1 in the figure) and click Explore. On the Explore page, select your resource from the Explore drop-down list (2 in the figure) to check whether metric data is pushed.