All Products
Search
Document Center

Alibaba Cloud Service Mesh:Enable distributed tracing in ASM

最終更新日:Aug 23, 2024

Service Mesh (ASM) integrates with Managed Service for OpenTelemetry. Managed Service for OpenTelemetry provides a wide range of tools to help you efficiently identify the performance bottlenecks of distributed applications. For example, you can use these tools to view trace data, display trace topologies, analyze application dependencies, and count the number of requests. This helps you improve the efficiency of developing and troubleshooting distributed applications. This topic describes how to enable distributed tracing in ASM.

Background information

Distributed tracing can be used to profile and monitor applications, especially those built using a microservices model. Sidecar proxies in an ASM instance can automatically generate and send spans. However, applications must propagate appropriate HTTP headers so that the spans sent by sidecar proxies can be correctly correlated with a single trace. When a sidecar proxy sends spans, the application must collect specific headers from each incoming request and forward the headers to all outgoing requests triggered by that incoming request. OpenTelemetry supports different propagators. Different propagators use different headers. The following section lists the propagators supported by OpenTelemetry and the corresponding headers.

B3 propagators

  • x-request-id

  • x-b3-traceid

  • x-b3-spanid

  • x-b3-parentspanid

  • x-b3-sampled

  • x-b3-flags

  • x-ot-span-context

W3C Trace Context propagators

  • traceparent

  • tracestate

Note

ASM instances of V1.18.0.124 or earlier use B3 propagators by default. ASM instances of versions later than V1.18.0.124 use W3C Trace Context propagators by default.

Prerequisites

Description of the sample application

In the Bookinfo book review application, the productpage service implemented in Python uses OpenTracing libraries and extracts the required headers from HTTP requests in B3 propagation format.

def getForwardHeaders(request):
    headers = {}
    # x-b3-*** headers can be populated using the opentracing span
    span = get_current_span()
    carrier = {}
    tracer.inject(
        span_context=span.context,
        format=Format.HTTP_HEADERS,
        carrier=carrier)
    headers.update(carrier)
    # ...

    incoming_headers = ['x-request-id']

    # ...

    for ihdr in incoming_headers:
        val = request.headers.get(ihdr)
        if val is not None:
            headers[ihdr] = val
    return headers

The reviews service implemented in Java also uses HTTP headers in B3 propagation format.

@GET
@Path("/reviews/{productId}")
public Response bookReviewsById(@PathParam("productId") int productId,
                            @HeaderParam("end-user") String user,
                            @HeaderParam("x-request-id") String xreq,
                            @HeaderParam("x-b3-traceid") String xtraceid,
                            @HeaderParam("x-b3-spanid") String xspanid,
                            @HeaderParam("x-b3-parentspanid") String xparentspanid,
                            @HeaderParam("x-b3-sampled") String xsampled,
                            @HeaderParam("x-b3-flags") String xflags,
                            @HeaderParam("x-ot-span-context") String xotspan) {

  if (ratings_enabled) {
    JsonObject ratingsResponse = getRatings(Integer.toString(productId), user, xreq, xtraceid, xspanid, xparentspanid, xsampled, xflags, xotspan);

Access example

Enter a URL in the format of http://{IP address of the ingress gateway}/productpage in the address bar of your browser, and then press the Enter key. You are navigated to the page of the Bookinfo application.

View applications

On the Applications page of the Managed Service for OpenTelemetry console, you can view the key metrics of all monitored applications, including the number of requests and errors in the current day and the health status. You can also configure labels for applications so that you can filter applications by label.

  1. Log on to the Managed Service for OpenTelemetry console.

  2. In the left-side navigation pane, click Applications. At the top of the Applications page, select a region as needed.应用列表

View application details

On the Application Details page of an application in the Managed Service for OpenTelemetry console, you can view the key metrics, call topology, and traces of the application on each server on which it is deployed.

  1. Log on to the Managed Service for OpenTelemetry console.

  2. In the left-side navigation pane, click Applications. At the top of the Applications page, select a region as needed. Then, click the name of the application that you want to check.

  3. In the left-side navigation pane, click Application Details. Then, click All or select a server, represented by an IP address, from the server list on the left.

    On the Overview tab, you can view the call topology and key metrics. You can also click the Traces tab to view the traces of the application on the selected server. A maximum of 100 traces are listed in descending order of the amount of time consumed.调用链路

View the waterfall chart of a trace

The waterfall chart of a trace shows the information of the trace, including the log generation time, status, IP address or server name, service name, and timeline.

  1. On the Application Details page of the application that you want to check, click the Traces tab. In the trace list, click the ID of the trace that you want to check.

  2. On the page that appears, view the waterfall chart of the trace.

    Note
    • The IP Address column may display IP addresses or server names. The information displayed depends on the display settings that are configured on the Application Settings page. For more information, see Manage applications and tags.

    • You can move the pointer over a service name to view the information of the service, including the duration, start time, tags, and log events. For more information, see Application details.

    瀑布图

FAQ

Why am I unable to view the trace information after I collect tracing data to Managed Service for OpenTelemetry in the ASM console?

  1. View the trace push logs.

    1. Obtain the kubeconfig file of the cluster and use kubectl to connect to the cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

    2. Run the following command to view the trace push logs of the pods related to the tracing-on-external-zipkin deployment in the istio-system namespace:

      kubectl logs "$(kubectl get pods -n istio-system  -l app=tracing -o jsonpath='{.items[0].metadata.name}')" -n istio-system -c  nginx

      The status code 406 is recorded in the trace push logs.日志

  2. View the quota on the number of service requests per day and the actual number of service requests on the previous day.

    1. Log on to the Managed Service for OpenTelemetry console.

    2. In the left-side navigation pane, click Cluster Configurations to view the quota on the number of service requests per day and the actual number of service requests on the previous day.

      额度配置

    You can find that the actual number of service requests on the previous day is greater than the quota on the number of service requests per day that you set in the Managed Service for OpenTelemetry console.

  3. Modify the quota on the number of service requests per day.

    If the actual number of service requests in a day is greater than the quota on the number of service requests per day, the reported data is discarded and no trace information is displayed. To resolve the issue, you must increase the quota on the number of service requests per day to make it greater than the actual number of service requests in a day.

    1. In the left-side navigation pane of the Managed Service for OpenTelemetry console, click Cluster Configurations.

    2. On the Cluster Configurations tab, modify the value of the Configuration quota parameter in the Quota configuration section. Increase the value to make it greater than the actual number of service requests in a day. Then, click Save.

    3. In the Cue message that appears, click OK.

Why is the traceid changed after the request is processed by the ingress gateway or sidecar?

The possible cause is that the propagation headers of the request initiated by your application are incomplete, and therefore, the sidecar proxy believes that trace information is incomplete and regenerates propagation headers. If you want to use a custom traceid, include at least two headers x-b3-trace-id and x-b3-spanid in your request in B3 propagation format and include at least a traceparent header in your request in W3C Trace Context propagation format.