Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog A Brief Look at OpenTelemetry

A Brief Look at OpenTelemetry

This article is a brief introduction to OpenTelemetry.

By Wupeng

1

OpenTelemetry is an observability project of the Cloud Native Computing Foundation (CNCF). It aims to provide a standardized solution for the field of observability, solving the standardization of data models, collection, processing, and export of observational data. Moreover, it also provides services unrelated to third-party vendors.

OpenTelemetry's tracing specification reached version 1.0 on February 10, 2021. Based on this milestone, OpenTelemetry will be explored in this article, and its value and development prospects in the observability field will be judged as well.

The following sections will explain OpenTelemetry.

What Is OpenTelemetry?

Taken from the What Is OpenTelemetry? page on the official website:

OpenTelemetry is a set of APIs, SDKs, tooling, and integrations that are designed for the creation and management of telemetry data, such as traces, metrics, and logs. New types of observational data may emerge in the future.

The project provides a vendor-agnostic implementation that can be configured to send telemetry data to the backend(s) of your choice. It supports a variety of popular open-source projects, including Jaeger and Prometheus.

OpenTelemetry is not an observability backend like Jaeger or Prometheus. Instead, it supports exporting data to a variety of open-source and commercial backends. It provides a pluggable architecture so additional technology protocols and formats can be easily added.

In short, OpenTelemetry does not provide backend services related to observability. These backend services usually provide services, such as storage, query, and visualization.

The following diagram can be used to understand the working scope of OpenTelemetry:

2

What Is the Problem Domain That OpenTelemetry Faces?

Wikipedia defines observability as, "In control theory, observability is a measure of how well internal states of a system can be inferred from knowledge of its external outputs."

Consider a physical system modeled in state-space representation. A system is said to be observable if, for any possible evolution of state and control vectors, the current state can be estimated using only the information from outputs. Physically, this generally corresponds to information obtained by sensors. In other words, one can determine the behavior of the entire system from the system's outputs. On the other hand, if the system is not observable, there are state trajectories that are not distinguishable by only measuring the outputs.

In summary, observability is a method to derive the internal state of the system from its external output.

The following diagram simplifies the system composition and interaction between systems:

3

From the interaction diagram above, the interaction behavior of the system has the following forms:

  • Internal System
  • Component features a closed loop that does not interact with other components or systems
  • Interaction between components
  • Between systems
  • Interaction between systems

As such, if one wants to know the status of the system through the external output of the system, two forms of information are needed:

  • Information of the component closed loop
  • Information flowing between components or systems

The first form can usually be characterized by logs or metrics, while the second form needs to be characterized by trace, adding flags in the flowing information.

The differences between logs and metrics can be understood through their operation methods.

Abstractly, observability involves the following problems:

  • Data model of Observational Data
  • Collection of Observational Data
  • Processing of Observational Data
  • Export of Observational Data
  • Use of Observational Data

These are problem domains and specific problems that OpenTelemetry faces, and the specific problems are limited in:

  • Data model of Observational Data
  • Collection of Observational Data
  • Processing of Observational Data
  • Export of Observational Data

What Are the Solutions of OpenTelemetry?

OpenTelemetry standardizes the data model of observational data through specification, and the collection, processing, and export methods, including trace, metrics, and logs. New types of observational data may emerge in the future. For more information, please see OpenTelemetry - specification.

Meanwhile, it is described by protocol buffers for ease of use. For more information, please see OpenTelemetry-proto.

Based on the Specification, OpenTelemetry has made the following efforts to generate and process observational data:

The following diagram shows the components and workflows of OpenTelemetry:

4

What Is the History of OpenTelemetry?

The information below is from CNCF's article entitled, A brief history of OpenTelemetry (So Far). OpenTelemetry is composed of two open-source projects:

  • OpenCensus – Standardizes data models for traces and metrics and provides tools for data collection, processing, and export
  • OpenTracing – Standardizes data models for traces and provides tools for data collection, processing, and export

The two open-source projects were merged and officially announced as the open-source OpenTelemetry project in May 2019.

Trace specification reached version 1.0 in February 2021. According to the official maturity model, the specification of trace has reached the stable level, and the specification of metrics has reached the beta level, but the specification of logs is still at the alpha level:

5

What Is the Prospect of OpenTelemetry?

More vendors have paid attention and contributed to OpenTelemetry since its launch.

From OpenTelemetry-Collector-contributions, you can see vendors have focused on the exporter, simply importing the observational data into their own services, which already includes Alibaba Cloud's SLS products:

6

It is believed that vendors will gradually put more effort into the receiver and processor. For example:

  • Form a workflow for processing observational data through the cooperation of receiver and exporter
  • Standardize observational data prior to storage with the processor

For multi-cloud scenarios, the observational data model and the standards of collection, processing, and export defined by OpenTelemetry allow users to connect to multiple cloud vendors through a set of observability standards to avoid vendor lock.

Even if services are provided for a single cloud service, such as internal services of cloud vendors, issues, such as future open-source and external co-construction will be inevitably considered. Open-source costs can also be reduced with observability standards from the community. Furthermore, the concepts, standards, and technologies of observability are evolving constantly. The technology benefits and impact brought by the community can be used better by following up with the community.

Therefore, it is necessary to adopt the industry's observability standards for multi-cloud scenarios and a single cloud vendor.

How Can I Use OpenTelemetry?

Core Concepts

OpenTelemetry has many concepts. Some common concepts are listed below:

• Related Observational Data

  • Signal

    • Types of observational data, such as trace, metrics, and logs

  • Instrument

    • Can be considered an instance of a certain kind of signal

• Related OpenTelemetry Projects

  • APIs

  • SDKs

    • API implementation for different development languages

  • Contribution Packages

    • Implementation related to specific open source projects or vendor products.

• Used Components

  • Components

    • Receivers

      • Components that receive observational data

    • Processors

      • Components that process received observational data

    • Exporters

      • Components that export observational data. For example, export observational data to the open-source project Prometheus or cloud vendor services

    • Extensions

      • It is not involved in the processing of observational data, but it can assist in the running of related processing components, such as health detection, service discovery.

    • Services

      • Characterize which components of the configuration need to be run, such as receivers, processors, exporters, and extensions

    • Collector

      • Can be considered as a whole consisting of receivers, processors, exporters, and extensions

    • Controller

      • Used in developer-developed applications, acting as a whole of receivers, processors, and exporters

Golang Demo

Here is a Golang demo that has been written to demonstrate:

  • How to generate trace and metrics data in applications
  • Data collection, processing, and printing of trace and metrics by standard output (stdout) controller in applications
  • A process of collecting trace and metrics data by the OpenTelemetry protocol (OTLP) in applications and then exporting to an external collector
  • A process of running a local standalone collector service to receive trace and metrics data pushed by the OTLP controller and to export it to local files and Alibaba Cloud Log Service (SLS)

For the demo, please see this GitHub link

For specific usage methods, please see the README.md section of the demo. The following is a brief description of the idea:

The cmd/app/server.go file describes the logic of OpenTelemetry and includes two parts:

  1. Initialize and run the global controller, which is used to receive, process, and export the observational data within the application or push such data to the outside
  2. Generate metrics and traces based on business requirements in applications

7

The pkg/ directory encapsulates the controller and signal (trace and metrics), respectively.

8

An example of exporting observational data to SLS is provided in yaml/, which includes the receiver for receiving observational data that the client can use to push data to the receiver through the gRPC client, processors for data conversion, exporters for data export, and services for enabling components:

9

Imagination

You will gain an intuitive experience of OpenTelemetry's concept, problem domain, solutions, and usage through the analysis above. Also, you can quick start OpenTelemetry through the Golang demo above.

For developers, OpenTelemetry can be used to generate and export traces, metrics, and logs. This reduces the cost of using different types of observational data during development and the cost of connecting to different backend services, such as the open-source project Prometheus or the services of third-party cloud vendors.

For SREs, OpenTelemetry can provide a set of standard data collection, processing, and export processes for observational data and standardize the data during the processing according to the team's requirements. By doing so, it is convenient for the subsequent use of standardized solutions, such as data monitoring and alarm.

Meanwhile, developers and SRE can use the community to continuously iterate their understanding of the observability problem domain and obtain the technical benefits. Then, they can provide feedback on the best practices generated in production to the community, promoting the development of the observability field.

References

You are welcome to leave a comment on the stability assurance issues while using Kubernetes and the stability assurance tools or services you are looking forward to using.

Container Trends Computing Log Service Resource Monitoring Kubernetes cloud native Golang OpenTelemetry
0 0 0
Share on

Read previous post:

Cloud-Native Provides More Possibilities for Developers

Read next post:

Go MySQL Driver Integrates Seata-Golang to Solve Distributed Transactions

Alibaba Cloud Native Community

506 posts | 48 followers

You may also like

Comments

Alibaba Cloud Native Community

506 posts | 48 followers

Related Products

More Posts by Alibaba Cloud Native Community

See All