Application Real-Time Monitoring Service:Use OpenTelemetry SDK for Java to add custom instrumentation code to traces

For information about the components and frameworks supported by ARMS, see Java components and frameworks supported by ARMS.

Add dependencies

Add the following Maven dependencies to introduce OpenTelemetry SDK for Java. For more information, see Instrumentation.

<dependencies>
    <dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-api</artifactId>
    </dependency>
    <dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-sdk-trace</artifactId>
    </dependency>
    <dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-sdk</artifactId>
    </dependency>
</dependencies>

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-bom</artifactId>
      <version>1.23.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

Compatibility between ARMS and OpenTelemetry-based instrumentation

ARMS is compatible with the following concepts involved in OpenTelemetry-based instrumentation. For more information about other concepts introduced by OpenTelemetry, see OpenTelemetry specification.

• Span: a specific operation in a request, such as a remote call or an internal method call.

• SpanContext: the context of a span. A SpanContext is associated with a specific operation in a request.

• Attribute: an additional attribute field of a span, which is used to record key information.

Spans in OpenTelemetry can be divided into the following types:

• Entry span: creates a SpanContext. Examples: Server and Consumer spans.

  Note

  For an entry span, the methods used for instrumentation in ARMS are often built in the framework, and the SpanContext already exists before your application is manually instrumented. ARMS records the entry span in OpenTelemetry as an internal span. For a method where no instrumentation is implemented in ARMS, such as a method for an asynchronous call or a custom RPC framework, the entry span in OpenTelemetry remains unchanged. ARMS performs aggregation on the client to generate relevant statistics.

• Internal span: reuses an existing SpanContext. An internal span is recorded as an internal method stack.

• Exit span: propagates a SpanContext. Examples: Client and Producer spans.

ARMS is compatible with the entry spans and internal spans of OpenTelemetry. However, ARMS does not allow exit spans to propagate SpanContexts based on the OpenTelemetry standard. In ARMS, exit spans must propagate SpanContexts based on the formats provided by ARMS.

The following code provides an example on how to create spans by using OpenTelemetry SDK for Java:

package com.alibaba.arms.brightroar.console.controller;

import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.baggage.Baggage;
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.api.trace.propagation.W3CTraceContextPropagator;
import io.opentelemetry.context.Context;
import io.opentelemetry.context.Scope;
import io.opentelemetry.context.propagation.ContextPropagators;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.PostConstruct;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

@RestController
@RequestMapping("/ot")
public class OpenTelemetryController {

    private Tracer tracer;

    private ScheduledExecutorService ses = Executors.newSingleThreadScheduledExecutor();

    @PostConstruct
    public void init() {
		OpenTelemetry openTelemetry = OpenTelemetrySdk.builder()
			.setPropagators(ContextPropagators.create(W3CTraceContextPropagator.getInstance()))
			.buildAndRegisterGlobal();

		tracer = GlobalOpenTelemetry.get().getTracer("manual-sdk", "1.0.0");

        ses.scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                Span span = tracer.spanBuilder("schedule")
                        .setAttribute("schedule.time", System.currentTimeMillis())
                        .startSpan();
                try (Scope scope = span.makeCurrent()) {
                    System.out.println("scheduled!");
                    Thread.sleep(500L);
                    span.setAttribute("schedule.success", true);
                    System.out.println(Span.current().getSpanContext().getTraceId()); // Obtain the trace ID.
                } catch (Throwable t) {
                    span.setStatus(StatusCode.ERROR, t.getMessage());
                } finally {
                    span.end();
                }
            }
        }, 10, 30, TimeUnit.SECONDS);
    }

    @ResponseBody
    @RequestMapping("/parent")
    public String parent() {
        Span span = tracer.spanBuilder("parent").setSpanKind(SpanKind.SERVER).startSpan();
        try (Scope scope = span.makeCurrent()) {
            // Use Baggage to propagate the custom tags of the service.
            Baggage baggage = Baggage.builder()
                    .put("user.id", "1")
                    .put("user.name", "name")
                    .build();
            try (Scope baggageScope = baggage.storeInContext(Context.current()).makeCurrent()) {
                child();
            }
            span.setAttribute("http.method", "GET");
            span.setAttribute("http.uri", "/parent");
        } finally {
            span.end();
        }
        return "parent";
    }

    private void child() {
        Span span = tracer.spanBuilder("child").startSpan();
        try (Scope scope = span.makeCurrent()) {
            span.setAttribute("user.id", Baggage.current().getEntryValue("user.id"));
            span.addEvent("Sleep Start");
            Thread.sleep(1000);
            Attributes attr = Attributes.of(AttributeKey.longKey("cost"), 1000L);
            span.addEvent("Sleep End", attr);
        } catch (Throwable e) {
            span.setStatus(StatusCode.ERROR, e.getMessage());
        } finally {
            span.end();
        }
    }

}

The sample code can be used to create the following three spans:

• parent: a parent span, which serves as the entry span of an HTTP request based on the OpenTelemetry standard. However, ARMS already creates a SpanContext in the embedded code of Apache Tomcat. Therefore, ARMS records the parent span as an internal method on the ARMS method stack.

• child: the internal span of the parent span. The internal span is recorded as an internal method on the ARMS method stack.

• schedule: an entry span for an independent thread. By default, ARMS does not create SpanContexts for schedule spans. Therefore, ARMS records a schedule span as a custom method and generates relevant statistics.

Use OpenTelemetry Baggage API to propagate the custom tags of a service

In OpenTelemetry, Baggage is contextual information that can be propagated across spans. You can configure key-value pairs for Baggage to propagate the custom tags of a service. Baggage is stored and propagated in HTTP headers. Therefore, we recommend that you do not store sensitive data in Baggage.

The parent span in the preceding sample code first stores two key-value pairs in Baggage, and then obtains the values that are stored in Baggage in the child span.

Obtain the trace ID

SpanContext contains a trace ID and a span ID. You can use the Span.current().getSpanContext().getTraceId() method to obtain the trace ID.

View the information about parent and child spans in the ARMS console

On the Method Stack tab in the Details panel of the /ot/parent HTTP method, you can view the information about parent and child spans. For more information, see Trace query.

View the information about a schedule span in the ARMS console

You can view the information about a schedule span on the following pages in the ARMS console:

• You can view the statistics on the schedule span on the Application Overview page in the ARMS console. For more information, see Application overview.

• You can view the details of the schedule span on the Interface Invocation page. For more information, see Interface calls.

• You can view the schedule span on the Traces tab. For more information, see Trace query.

  On the Traces tab, click the Zoom In icon to view the details of the schedule span.

  ARMS displays the attributes of an OpenTelemetry span as tags. To view the attributes of a span, click the span name.

  

Asynchronously pass the context

The following sample code describes an event. When the event is produced, the trace context of the producer thread is recorded to the event. When the event is consumed, the context is fetched and restored.

class Event {
    private Context context;
    private String msg;

    public Event(Context context, String msg) {
        this.context = context;
        this.msg = msg;
    }
}

private final LinkedBlockingQueue<Event> linkedBlockingQueue = new LinkedBlockingQueue<Event>();

public void produce(String msg) {
    linkedBlockingQueue.add(new Event(Context.current(), msg));
}

public void consume() throws Exception {
    Event event = linkedBlockingQueue.take();
    try(Scope scope = event.context.makeCurrent()) {
        processEvent(event);
    }
}

public void processEvent(Event event) {
    //todo process event
}

References

You can associate trace IDs with the business logs of an application. This way, if an error occurs in the application, you can access the business logs associated with trace IDs to find out and troubleshoot the error. For more information, see Associate trace IDs with business logs.