Alibaba Cloud Service Mesh:Enable distributed tracing in ASM

Description of the sample application

def getForwardHeaders(request):
    headers = {}
    # x-b3-*** headers can be populated using the opentracing span
    span = get_current_span()
    carrier = {}
    tracer.inject(
        span_context=span.context,
        format=Format.HTTP_HEADERS,
        carrier=carrier)
    headers.update(carrier)
    # ...

    incoming_headers = ['x-request-id']

    # ...

    for ihdr in incoming_headers:
        val = request.headers.get(ihdr)
        if val is not None:
            headers[ihdr] = val
    return headers

@GET
@Path("/reviews/{productId}")
public Response bookReviewsById(@PathParam("productId") int productId,
                            @HeaderParam("end-user") String user,
                            @HeaderParam("x-request-id") String xreq,
                            @HeaderParam("x-b3-traceid") String xtraceid,
                            @HeaderParam("x-b3-spanid") String xspanid,
                            @HeaderParam("x-b3-parentspanid") String xparentspanid,
                            @HeaderParam("x-b3-sampled") String xsampled,
                            @HeaderParam("x-b3-flags") String xflags,
                            @HeaderParam("x-ot-span-context") String xotspan) {

  if (ratings_enabled) {
    JsonObject ratingsResponse = getRatings(Integer.toString(productId), user, xreq, xtraceid, xspanid, xparentspanid, xsampled, xflags, xotspan);

Access example

Enter a URL in the format of http://{IP address of the ingress gateway}/productpage in the address bar of your browser, and then press the Enter key. You are navigated to the page of the Bookinfo application.

View applications

On the Applications page of the Managed Service for OpenTelemetry console, you can view the key metrics of all monitored applications, including the number of requests and errors in the current day and the health status. You can also configure labels for applications so that you can filter applications by label.

1. Log on to the Managed Service for OpenTelemetry console.

2. In the left-side navigation pane, click Applications. At the top of the Applications page, select a region as needed.

View application details

On the Application Details page of an application in the Managed Service for OpenTelemetry console, you can view the key metrics, call topology, and traces of the application on each server on which it is deployed.

1. Log on to the Managed Service for OpenTelemetry console.

2. In the left-side navigation pane, click Applications. At the top of the Applications page, select a region as needed. Then, click the name of the application that you want to check.

3. In the left-side navigation pane, click Application Details. Then, click All or select a server, represented by an IP address, from the server list on the left.

   On the Overview tab, you can view the call topology and key metrics. You can also click the Traces tab to view the traces of the application on the selected server. A maximum of 100 traces are listed in descending order of the amount of time consumed.

View the waterfall chart of a trace

The waterfall chart of a trace shows the information of the trace, including the log generation time, status, IP address or server name, service name, and timeline.

1. On the Application Details page of the application that you want to check, click the Traces tab. In the trace list, click the ID of the trace that you want to check.

2. On the page that appears, view the waterfall chart of the trace.

   Note

   • The IP Address column may display IP addresses or server names. The information displayed depends on the display settings that are configured on the Application Settings page. For more information, see Manage applications and tags.

   • You can move the pointer over a service name to view the information of the service, including the duration, start time, tags, and log events. For more information, see Application details.

   

FAQ

Why am I unable to view the trace information after I collect tracing data to Managed Service for OpenTelemetry in the ASM console?

1. View the trace push logs.

   1. Obtain the kubeconfig file of the cluster and use kubectl to connect to the cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

   2. Run the following command to view the trace push logs of the pods related to the tracing-on-external-zipkin deployment in the istio-system namespace:

      kubectl logs "$(kubectl get pods -n istio-system  -l app=tracing -o jsonpath='{.items[0].metadata.name}')" -n istio-system -c  nginx

      The status code 406 is recorded in the trace push logs.

2. View the quota on the number of service requests per day and the actual number of service requests on the previous day.

   1. Log on to the Managed Service for OpenTelemetry console.

   2. In the left-side navigation pane, click Cluster Configurations to view the quota on the number of service requests per day and the actual number of service requests on the previous day.

      

   You can find that the actual number of service requests on the previous day is greater than the quota on the number of service requests per day that you set in the Managed Service for OpenTelemetry console.

3. Modify the quota on the number of service requests per day.

   If the actual number of service requests in a day is greater than the quota on the number of service requests per day, the reported data is discarded and no trace information is displayed. To resolve the issue, you must increase the quota on the number of service requests per day to make it greater than the actual number of service requests in a day.

   1. In the left-side navigation pane of the Managed Service for OpenTelemetry console, click Cluster Configurations.

   2. On the Cluster Configurations tab, modify the value of the Configuration quota parameter in the Quota configuration section. Increase the value to make it greater than the actual number of service requests in a day. Then, click Save.

   3. In the Cue message that appears, click OK.

Why is the traceid changed after the request is processed by the ingress gateway or sidecar?

The possible cause is that the propagation headers of the request initiated by your application are incomplete, and therefore, the sidecar proxy believes that trace information is incomplete and regenerates propagation headers. If you want to use a custom traceid, include at least two headers x-b3-trace-id and x-b3-spanid in your request in B3 propagation format and include at least a traceparent header in your request in W3C Trace Context propagation format.