How do I export monitoring data in real time? - CloudMonitor

CloudMonitor provides the cloud monitoring exporter feature to help you export monitoring data in the cloud.

Prerequisites

Hybrid Cloud Monitoring is activated. For more information, see Activate Hybrid Cloud Monitoring.

Background information

You are not charged for activating Hybrid Cloud Monitoring. You are charged based on the actual resource usage. For information about the billing of the cloud monitoring exporter feature, see Billing of the cloud monitoring exporter feature.

Procedure

Call the DescribeMetricMetaList operation to obtain the metrics for each Alibaba Cloud service.

For information about how to set the parameters of the DescribeMetricMetaList operation, see DescribeMetricMetaList.

Note

For information about the Namespace, MetricName, Dimensions, Period, and Statistics parameters of the metrics for each Alibaba Cloud service, see Appendix 1: Metrics.

Use Maven to import the CloudMonitor SDK.

<dependency>
  <groupId>com.aliyun</groupId>
  <artifactId>cms20190101</artifactId>
  <version><Latest version of the SDK package></version>
</dependency>

Call the DescribeMetricMetaList operation to obtain the metrics for each Alibaba Cloud service.

The following code shows a sample SDK request:

private static Client getClient() throws Exception {
    Config config = new Config()
    config.setAccessKeyId(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"));
  	config.setAccessKeySecret(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
    config.endpoint = "metrics.cn-hangzhou.aliyuncs.com";
    return new Client(config);
}
public static void main(String[] args) throws Exception {
    Client client = getClient();
    DescribeMetricMetaListRequest describeMetricMetaListRequest = new DescribeMetricMetaListRequest();
    describeMetricMetaListRequest.setNamespace("acs_ecs_dashboard");
    DescribeMetricMetaListResponse describeMetricMetaListResponse = client.describeMetricMetaList(describeMetricMetaListRequest);
    String json = JSON.toJSONString(describeMetricMetaListResponse.getBody());
    System.out.println(json);
}

Obtain metrics.

Use POM to import the CloudMonitor SDK that is used to obtain monitoring data.

<dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>cms20190101</artifactId>
    <version><Latest version of the SDK package></version>
</dependency>

Call the Cursor operation to specify the time range of the monitoring data that you want to export. The operation returns the value of the Cursor parameter. The value is required when you call the BatchExport operation for the first time.

For information about how to set the parameters of the Cursor operation, see Cursor.

Cursor and BatchExport are supported in all regions. For more information, see Endpoints.

The following code shows a sample SDK request:

private static Config config = null;
private static Config getConfig(Config config){
  config = new Config();
  config.setAccessKeyId(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"));
  config.setAccessKeySecret(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
  config.setEndpoint("metrics.cn-hangzhou.aliyuncs.com");
  return config;
}
private static String getCursor() throws Exception {
    Client client = new Client(getConfig(config));
    CursorRequest cursorRequest = new CursorRequest();
    cursorRequest.setNamespace("acs_ecs_dashboard");
    cursorRequest.setMetric("CPUUtilization");
    cursorRequest.setPeriod(60);
    cursorRequest.setStartTime(System.currentTimeMillis() - 60 * 60 * 1000);
    cursorRequest.setEndTime(Long.MAX_VALUE);
  	List<Matcher> matchers = new ArrayList<>();
  	Matcher matcher = new Matcher();
  	matcher.setLabel("userId");
    matcher.setValue("173651113438****");
  	matcher.setOperator("EQUALS");
  	matchers.add(matcher);
    cursorRequest.setMatchers(matchers);
    CursorResponse cursorResponse = client.cursor(cursorRequest);
    if (cursorResponse == null || cursorResponse.getBody() == null || cursorResponse.getBody().getCursor() == null) {
      return null;
    }
    return cursorResponse.getBody().getCursor();
}

An SDK response that is similar to the following code is returned:

eyJidWNrZXRzIjo0LCJlbmRUaW1lIjo5MjIzMzcyMDM2ODU0Nzc1ODA3LCJleHBvcnRFbmRUaW1lIjo5MjIzMzcyMDM2ODU0Nzc1ODA3LCJleHByZXNzUmFuZ2UiOmZhbHNlLCJoYXNOZXh0Ijp0cnVlLCJsaW1pdCI6NTAwMCwibG9nVGltZU1vZGUiOnRydWUsIm1hdGNoZXJzIjp7ImNoYWluIjpbeyJsYWJlbCI6InVzZXJJZCIsIm9wZXJhdG9yIjoiRVFVQUxTIiwidmFsdWUiOiIxNzM2NTExMTM0Mzg5MTEwIn0seyJsYWJlbCI6InRhc2tJZCIsIm9wZXJhdG9yIjoiRVFVQUxTIiwidmFsdWUiOiIqIn1dfSwibWV0cmljIjoiQXZhaWxhYmlsaXR5X29yaWdpbmFsIiwibWV0cmljVHlwZSI6Ik1FVFJJQyIsIm5hbWVzcGFjZSI6ImFjc19uZXR3b3JrbW9uaXRvciIsIm5leHRQa0FkYXB0ZXIiOnt9LCJvZmZzZXQiOjAsInN0YXJ0VGltZSI6MTYzODQwNTA1MDcxNywic3RlcCI6LTEsInRpbWVvdXQiOjEyMCwid2luZG93Ijo2****

Call the BatchExport operation to export the monitoring data that is defined in the Cursor operation.

For information about how to set the parameters of the BatchExport operation, see BatchExport.

The following code shows a sample SDK request:

private static String BatchExport(String cursor) throws Exception {
    Client client = new Client(getConfig(config));
    BatchExportRequest request = new BatchExportRequest();
    request.setNamespace("acs_ecs_dashboard");
    request.setMetric("CPUUtilization");
    request.setCursor(cursor);
    request.setLength(1000);
    BatchExportResponse response = client.batchExport(request);
    if (response == null || response.getBody() == null) {
      return null;
    }else {
      String json = JSON.toJSONString(response.getBody());
      return json;
    }
}

An SDK response that is similar to the following code is returned:

{
    "anchor":"168801465****",
    "code":200,
    "cursor":"eyJidWNrZXdWNrZXRCeXRlcyI6IndBPT0iLCJidWNrZXRzIjo0LCJjdXJyZW50QnVja2V0IjotNjQsImN1cnJlbnRUYXJnZXRUaW1lU2xvdCI6MTY4Nzk5NjgwMCwiZW5kVGltZSI6MTY4ODAxNDgwMDAwMCwiZXhwb3J0RW5kVGltZSI6MTY4ODAxODMxNTU0NywiZXhwb3J0U3RhcnRUaW1lIjoxNjg4MDE0NjU1NTQ3LCJleHByZXNzUmFuZ2UiOmZhbHNlLCJoYXNOZXh0Ijp0cnVlLCJpbmRleCI6MCwibGF0ZXN0TG9nVGltZSI6MCwibWF0Y2hlcnMiOnsiY2hhaW4iOlt7ImxhYmVsIjoidXNlcklkIiwib3BlcmF0b3IiOiJFUVVBTFMiLCJ2YWx1ZSI6IjE3MzY1MTExMzQzODkxMTAifV0sImxvY2tlZCI6dHJ1ZX0sIm1ldHJpYyI6IlNwbGl0cndQcm94eU1heFJlc3BvbnNlU2l6ZSIsIm1ldHJpY1R5cGUiOiJNRVRSSUMiLCJuYW1lc3BhY2UiOiJhY3Nfa3ZzdG9yZV9leCIsIm5leHRQa0FkYXB0ZXIiOnt9LCJvY2MiOlstNjU2MDE3MDUyMjc3MzE4NjY0LDE2ODgwMTgyNTk4MTNdLCJvZmZzZXQiOjEsIm9mZnNldERpZ2l0Ijo1NjgyMjY5NDU3MTc0NTM5ODk1LCJzdGFydFRpbWUiOjE2ODgwMTQ2NTU1NDcsInN0ZXAiOi0xLCJ0YWdNYXRjaGVycyI6eyJjaGFpbiI6W10sImxvY2tlZCI6dHJ1ZX0sInRhcmdldFRpbWVTbG90cyI6WzE2ODc5MTA0MDAsMTY4Nzk5NjgwMF0sInV1aWQiOiJjYjQzNjBjNS00NjY4LTQxZDQtYmRjYi0zMGNhY2M2NDBhNmQiLCJ2Y2MiOlsxMDc3NTkwNTEzMzk4NzczMzk3LDE2ODgwMTgyNTMz****",
    "dataResults":[ 
        {
            "associated":{

            },
            "dimensions":[
                {
                    "label":"userId",
                    "value":"173651113438****"
                },
                {
                    "label":"instanceId",
                    "value":"r-bp12meeb25mb7d****"
                },
                {
                    "label":"nodeId",
                    "value":"r-bp12meeb25mb7d****-proxy-3"
                }
            ],
            "measurements":{
                "Maximum":565.916,
                "Average":565.916
            },
            "metric":"SplitrwProxyMaxResponseSize",
            "namespace":"acs_kvstore",
            "period":60,
            "timestamp":1688014560000
        }
    ],
    "hasNext":true,
    "length":1, 
    "message":"success",  
    "requestId":"8D26C6ED-991C-533E-B601-B020EDF7****",
    "success":true
}

Examples

Example 1: Export monitoring data for the Availability metric of the acs_networkmonitor namespace in site monitoring at a regular interval.

Obtain the value of the Cursor parameter.

public class GetDataUtil {
    private static Config config = null;
		private static Config getConfig(Config config){
  	config = new Config();
  	config.setAccessKeyId(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"));
  	config.setAccessKeySecret(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
  	config.setEndpoint("metrics.cn-hangzhou.aliyuncs.com");
  	return config;
	}
	private static String getCursor() throws Exception {
		Client client = new Client(getConfig(config));
 	  CursorRequest cursorRequest = new CursorRequest();
 	  cursorRequest.setNamespace("acs_ecs_dashboard");
    cursorRequest.setMetric("CPUUtilization");
    cursorRequest.setPeriod(60);
    cursorRequest.setStartTime(System.currentTimeMillis() - 60 * 60 * 1000);
    cursorRequest.setEndTime(Long.MAX_VALUE);
  	List<Matcher> matchers = new ArrayList<>();
  	Matcher matcher = new Matcher();
  	matcher.setLabel("userId");
    matcher.setValue("173651113438****");
  	matcher.setOperator("EQUALS");
  	matchers.add(matcher);
    cursorRequest.setMatchers(matchers);
    CursorResponse cursorResponse = client.cursor(cursorRequest);
    if (cursorResponse == null || cursorResponse.getBody() == null || cursorResponse.getBody().getCursor() == null) {
      return null;
    }
    return cursorResponse.getBody().getCursor();
	}

    // ...
}

Export monitoring data.

public class GetDataUtil {
    // ...

    private static String BatchExport(String cursor) throws Exception {
    	Client client = new Client(getConfig(config));
    	BatchExportRequest request = new BatchExportRequest();
    	request.setNamespace("acs_ecs_dashboard");
    	request.setMetric("CPUUtilization");
    	request.setCursor(cursor);
    	request.setLength(1000);
    	BatchExportResponse response = client.batchExport(request);
    	if (response == null || response.getBody() == null) {
      	return null;
    	}else {
      	String json = JSON.toJSONString(response.getBody());
      	return json;
    	}
	}
}

Export monitoring data for the Availability metric at a regular interval.

For example, if you set the interval to 3 seconds, monitoring data is exported every 3 seconds.

public class Demo1 {
    private String cursor;
    @Scheduled(cron = "*/3 * * * * ?")
    public void getCpuData() throws Exception {
        System.out.println("Start to obtain data.");
        if(cursor == null){
            Matcher matcher = new Matcher();
            matcher.setLabel("userId").setOperator("EQUALS").setValue("173651113438****");
            List<Matcher> chain  = new ArrayList<>();
            chain.add(matcher);
            cursor = GetDataUtil.getCursor("acs_networkmonitor","Availability",60,System.currentTimeMillis() - 60 * 60 * 1000,Long.MAX_VALUE,chain);
        }
        String sth = GetDataUtil.BatchExport(cursor,"acs_networkmonitor","Availability",10);
        JSONObject jsonObject = JSONObject.parseObject(sth);
        if(!StringUtils.isEmpty(jsonObject.getString("cursor"))){
            cursor = jsonObject.getString("cursor");
        }
        System.out.println("Stop obtaining data.");
    }
}

Example 2: Export monitoring data for the Availability and Availability_new metrics of the acs_networkmonitor namespace in site monitoring at a regular interval.

Obtain the value of the Cursor parameter.

public class GetDataUtil {
    private static Config config = null;
		private static Config getConfig(Config config){
  	config = new Config();
  	config.setAccessKeyId(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"));
  	config.setAccessKeySecret(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
  	config.setEndpoint("metrics.cn-hangzhou.aliyuncs.com");
  	return config;
	}
	private static String getCursor() throws Exception {
		Client client = new Client(getConfig(config));
 	  CursorRequest cursorRequest = new CursorRequest();
 	  cursorRequest.setNamespace("acs_ecs_dashboard");
    cursorRequest.setMetric("CPUUtilization");
    cursorRequest.setPeriod(60);
    cursorRequest.setStartTime(System.currentTimeMillis() - 60 * 60 * 1000);
    cursorRequest.setEndTime(Long.MAX_VALUE);
  	List<Matcher> matchers = new ArrayList<>();
  	Matcher matcher = new Matcher();
  	matcher.setLabel("userId");
    matcher.setValue("173651113438****");
  	matcher.setOperator("EQUALS");
  	matchers.add(matcher);
    cursorRequest.setMatchers(matchers);
    CursorResponse cursorResponse = client.cursor(cursorRequest);
    if (cursorResponse == null || cursorResponse.getBody() == null || cursorResponse.getBody().getCursor() == null) {
      return null;
    }
    return cursorResponse.getBody().getCursor();
	}

    // ...
}

Export monitoring data.

public class GetDataUtil {
    // ...

    private static String BatchExport(String cursor) throws Exception {
    	Client client = new Client(getConfig(config));
    	BatchExportRequest request = new BatchExportRequest();
    	request.setNamespace("acs_ecs_dashboard");
    	request.setMetric("CPUUtilization");
    	request.setCursor(cursor);
    	request.setLength(1000);
    	BatchExportResponse response = client.batchExport(request);
    	if (response == null || response.getBody() == null) {
      	return null;
    	}else {
      	String json = JSON.toJSONString(response.getBody());
      	return json;
    	}
	}
}

Export monitoring metric for the Availability and Availability_new metrics at a regular interval.

For example, if you set the interval to 3 seconds, monitoring data is exported every 3 seconds.

public class Demo2 {
    private String availabilityCursor;
    private String availability_newCursor;
    @Scheduled(cron = "*/3 * * * * ?")
    public void getCpuData() throws Exception {
        System.out.println("Start to obtain data.");
        if(availabilityCursor == null){
            Matcher matcher = new Matcher();
            matcher.setLabel("userId").setOperator("EQUALS").setValue("173651113438****");
            List<Matcher> chain  = new ArrayList<>();
            chain.add(matcher);
            availabilityCursor = GetDataUtil.getCursor("acs_networkmonitor","Availability",60,System.currentTimeMillis() - 60 * 60 * 1000,Long.MAX_VALUE,chain);
        }
        if(availability_newCursor == null){
            Matcher matcher = new Matcher();
						matcher.setLabel("userId").setOperator("EQUALS").setValue("173651113438****");
            List<Matcher> chain  = new ArrayList<>();
            chain.add(matcher);
            availability_newCursor = GetDataUtil.getCursor("acs_networkmonitor","Availability_new",60,System.currentTimeMillis() - 60 * 60 * 1000,Long.MAX_VALUE,chain);
        }
        String sth1 = GetDataUtil.BatchExport(availabilityCursor,"acs_networkmonitor","Availability",10);
        String sth2 = GetDataUtil.BatchExport(availability_newCursor,"acs_networkmonitor","Availability_new",10);
        JSONObject jsonObject1 = JSONObject.parseObject(sth1);
        if(!StringUtils.isEmpty(jsonObject1.getString("cursor"))){
            availabilityCursor = jsonObject1.getString("cursor");
        }
        System.out.println("availability");
        System.out.println(sth1);
        JSONObject jsonObject2 = JSONObject.parseObject(sth2);
        if(!StringUtils.isEmpty(jsonObject2.getString("cursor"))){
            availability_newCursor = jsonObject2.getString("cursor");
        }
        System.out.println("availability_new");
        System.out.println(sth2);
        System.out.println("Stop obtaining data.");
    }
}