Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Prometheus: How to Monitor the Activity of a Service Running on Alibaba Cloud and How SLS Supports It

Prometheus: How to Monitor the Activity of a Service Running on Alibaba Cloud and How SLS Supports It

Here shows ways to monitor the activity of a service running on Alibaba Cloud with Prometheus and Blackbox Exporter and how SLS supports Prometheus.

Using Prometheus and Blackbox Exporter to Monitor Services on Alibaba Cloud

In this tutorial, we will explore ways to monitor the activity of a service running on an Alibaba Cloud ECS server. The tools for the job include Prometheus, Alertmanager and Blackbox Exporter. Follow this link to access the latest Prometheus client download. Developers and system managers use Prometheus to log system measurements, control resources, and archive the information in a time-series database. For alerting recipients, it uses an Alertmanager, which may replicate or group separate or identical alert forms. The boss has the benefit of being compatible with communications systems like email.

While Prometheus is a useful tool all by itself, it becomes even handier when integrated with an exporter. Your infrastructure can be separated into various components including web and database servers, messaging systems or APIs. An exporter such as Blackbox Exporter can have alerts configured for all these individual parts. Blackbox Exporter uses several techniques to run request metrics including HTTP, HTTPS, DNS, TCP, and ICMP. For each request, there are detailed data points such as the success of requests and response times. We shall use both tools to see how well a NGINX server running on Alibaba Cloud responds to our requests. We will also create email alerts if the server fails a response test.

What You Will Need for This Tutorial

To complete the setup guide in this tutorial, you should have the following:

  1. An Alibaba Cloud Elastic Compute Service (ECS) instance that has Ubuntu 18.04 installed on it.
  2. A NGINX server that is installed with Ubuntu.
  3. A server block listening through a known port. For this, we will use a Django app running on port 8000.
  4. Prometheus installed and running on your server. Steps to do this are given below.
  5. An SMTP email server

We will all of these set up in this tutorial. Note that it is also possible to configure Alertmanager to send notifications to Slack.

Cloud-Native Prometheus Solution: High Performance, High Availability, and Zero O&M

Alibaba Cloud Log Service (SLS) strives to develop itself into a DevOps data mid-end that provides rich capabilities including host data access, storage, analysis, and visualization. This article describes how SLS supports the Prometheus solution to provide a cloud-native Prometheus engine that features high performance, high availability, and zero O&M.

Prometheus - De-facto Standard for Cloud-Native Monitoring

Cloud-native technologies have been booming and flourishing across the world in recent years, and Cloud Native Computing Foundation (CNCF), one of the most influential projects in the IT field, is the strong support behind cloud-native technologies. As a non-profit organization under Linux Foundation, CNCF manages a dozen projects related to cloud-native technologies, among which the best known is Kubernetes, the de-facto standard in the container orchestration field.

Prometheus is the second CNCF graduated project, and has become the most popular one apart from Kubernetes. It is no exaggeration to say that Prometheus has become a de-facto standard of cloud-native monitoring. If the first step of enabling cloud native is to build a Kubernetes environment, then Prometheus is the first step to implement cloud-native monitoring.

Prometheus

After you deploy apps in Kubernetes, you will find it necessary to check the running statuses of the cluster and the apps. However, some of the monitoring methods in the virtual machine (VM) environment are no longer applicable. Although there are several alternatives to Prometheus, it is the best choice for many applications due to these advantages:

  1. Prometheus is easy to deploy. Especially in the Kubernetes environment. You only need several YAML files to configure Prometheus and its monitoring items after Prometheus Operator is deployed. Then, you will gain overall information about the monitored products in the monitoring dashboard by using Grafana and its rich set of Prometheus templates.
  2. Prometheus has a wealth of service discovery mechanisms. In particular, it can collect Kubernetes pod indicators by only declaring one simple annotation.
  3. Prometheus' exporters cover almost all open-source software systems and are supported by many commercial software and systems, such as Alibaba Cloud CloudMonitor which provides a Prometheus Exporter module.
  4. Prometheus provides software development kits (SDKs) for almost all languages so that you can expose metrics in an app. These SDKs are elegantly designed and convenient to expose metrics.
  5. Prometheus is an open-source project under CNCF. Therefore, you do not need to worry that the software updates will be stopped in a few years.
  6. If you take a closer look at Kubernetes code, you will find that all Kubernetes components expose Prometheus metrics, and that Prometheus is indispensable for monitoring Kubernetes.

SLS and Cloud-native Technologies

Alibaba Cloud Log Service (SLS) strives to develop itself into a DevOps data mid-end that provides rich capabilities such as host data access, storage, analysis, and visualization. It provides an all-in-one platform where you can easily handle data-related tasks in DevOps scenarios and quickly build your enterprise's observable platform.

Alibaba Cloud Log Service

SLS provides a wide range of data access methods and supports many data access approaches related to cloud-native observability. The preceding figure shows the projects that are supported by SLS for data access in the CNCF landscape. The monitoring, logging, and tracing features all support CNCF graduated projects, such as Prometheus, Fluentd, and Jaeger. The main reasons for using SLS to store Prometheus monitoring data include:

  1. SLS data can be stored persistently. Many users want to store key Prometheus metrics persistently in SLS.
  2. Many users now store their logging and tracing data in SLS and want to do so for Prometheus data as well, so as to implement integrated observability data solutions and reduce O&M workloads.
  3. SLS provides many metric-related AIOps algorithms, such as multi-cycle estimation, prediction, outlier detection, and time series classification. Clients also expect more intelligent use of Prometheus data.
  4. SLS also supports data pipeline models. Prometheus can enable faster alarming if its metrics are interconnected with downstream systems for stream computing. In addition, Prometheus can enable offline statistical analysis if its metrics are interconnected with data warehouses.

SLS Solutions for Prometheus

The SLS MetricStore provides native support for PromQL. All data is distributed to multiple hosts for distributed storage as shards. The computing layer integrates a Prometheus QueryEngine module to separate storage and computing, so that massive data processing can be carried out easily.

SLS Solutions for Prometheus

Compared with the community-provided Prometheus distributed extensions, such as Cortex, Thanos, M3DB, FiloDB, and VictoriaMetrics, the SLS's distributed implementation solution is closer to the community's goal of solving the restrictions on the use of native Prometheus.

  1. Compatibility: SLS implementation reuses the Prometheus code without any modification needed. This ensures the SLS implementation keeps pace with the official updates in real time.
  2. Global view: SLS is an SaaS-based service and supports multitenancy and multiple instances. Therefore, it can write the data of multiple clusters to the same instance to display a global view.
  3. Persistent storage: SLS data supports the TTL mechanism and persistent storage.
  4. High availability: Each instance contains multiple shards, and different shards are allocated to different hosts. The failure of hosts where some shards are stored does not compromise the overall writing performance. Each shard has three replicas on Apsara Distributed File System to ensure the reliability of each shard.

In addition to supporting these requirements of the community, SLS can provide the following advantages for Prometheus:

  1. Larger storage: SLS is a fully cloud-based service. The storage space for each user is unlimited.
  2. Lower costs: In terms of labor cost, SLS's Prometheus access method does not require the operation and maintenance of Prometheus instances. In terms of usage, SLS MetricStore uses a pay-as-you-go model without the need to separately purchase hosts and disks for data calculation and storage.
  3. Faster speed: The storage and computing separation architecture of SLS gives full play to cluster capabilities, enabling faster end-to-end processing especially in processing of massive data.
  4. More intelligent algorithms: All of SLS's metric-related AI algorithms can be applied to Prometheus data, such as multi-cycle estimation, prediction, outlier detection, and time series classification, to add AI power to Prometheus.
  5. More extensive ecosystems: SLS features sound connectivity with upstream and downstream ecosystems. Therefore, you can integrate Prometheus metrics with stream computing for faster alarming, with data warehouses for offline statistical analysis, and with OSS for archival storage.
  6. Better support: Observability requires full connectivity between metrics, logging data, and tracing data. SLS is committed to building a unified OpenTelemetry storage platform to act as an underlying data foundation for all kinds of intelligent data apps.

Related Products

Elastic Compute Service

Alibaba Cloud Elastic Compute Service (ECS) provides fast memory and the latest Intel CPUs to help you to power your cloud applications and achieve faster results with low latency.

Log Service

Log Service is a complete real-time data logging service that has been developed by Alibaba Group. Log Service supports collection, consumption, shipping, search, and analysis of logs, and improves the capacity of processing and analyzing large amounts of logs.

Related Documentations

What is Log Service?

Log Service (SLS) is a one-stop logging service developed by Alibaba Cloud that is widely used by Alibaba Group in big data scenarios. You can use Log Service to collect, query, and consume log data without the need to invest in in-house data collection and processing systems. This enables you to focus on your business, improving business efficiency and helping your business to expand.

Features of Log Service

This topic describes the features of Log Service.

prometheus port monitoring prometheus blackbox exporter blackbox exporter modules blackbox exporter tcp blackbox yml
0 0 0
Share on

Read previous post:

Alibaba Cloud Red Hat OpenShift Container Platform 4.6 Deployment Guide

Read next post:

Multi-Tier Approach With Virtual Private Cloud – Part 2

Alibaba Clouder

2,599 posts | 764 followers

You may also like

Comments

Alibaba Clouder

2,599 posts | 764 followers

Related Products

More Posts by Alibaba Clouder

See All