Container Service for Kubernetes (ACK) allows you to enable tracing for the NGINX Ingress controller and import the trace data to Managed Service for OpenTelemetry. Managed Service for OpenTelemetry persists the trace data and aggregates and computes the trace data in real time to generate monitoring data, which includes trace details and real-time topology. You can troubleshoot and diagnose issues based on the monitoring data.
Prerequisites
The NGINX Ingress controller is installed. For more information, see Manage the NGINX Ingress controller.
Version description
The support for tracing varies based on the NGINX Ingress controller version. The following table describes the NGINX Ingress controller versions that support and do not support tracing.
NGINX Ingress controller version | OpenTelemetry | OpenTracing |
≥ 1.10.2-aliyun.1 | Supported | Not supported |
v1.9.3-aliyun.1 | Supported | Supported |
v1.8.2-aliyun.1 | Supported | Supported |
< v1.8.2-aliyun.1 | Not supported | Supported |
Procedure
Perform the following steps based on the version of the NGINX Ingress controller installed in your cluster.
OpenTelemetry
Step 1: Obtain an endpoint from Managed Service for OpenTelemetry
New version of the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console. In the left-side navigation pane, click Integration Center.
In the Open Source Frameworks section, click the OpenTelemetry card.
In the OpenTelemetry panel, select the region from which you want to import trace data.
Record the endpoint used to import data over gRPC.
NoteThe NGINX Ingress controller is deployed on Alibaba Cloud and resides in the same region as the Managed Service for OpenTelemetry agent in this example. Therefore, we recommend that you use a virtual private cloud (VPC) endpoint. If the NGINX Ingress controller and the Managed Service for OpenTelemetry agent are deployed in different regions, use a public endpoint.
Previous version of the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console.
In the left-side navigation pane, click Cluster Configurations. On the right side of the page that appears, click the Access point information tab.
In the upper part of the page, select the region from which you want to import trace data.
Select Show Token in the Cluster Information section and click OpenTelemetry in the Client section. Then, record the endpoint used to import data over gRPC.
NoteThe NGINX Ingress controller is deployed on Alibaba Cloud and resides in the same region as the Managed Service for OpenTelemetry agent in this example. Therefore, we recommend that you use a VPC endpoint. If the NGINX Ingress controller and the Managed Service for OpenTelemetry agent are deployed in different regions, use a public endpoint.
Step 2: Enable Managed Service for OpenTelemetry for the NGINX Ingress controller
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, choose .
In the upper part of the page, select kube-system from the Namespace drop-down list. Then, enter nginx-ingress-controller in the search box and click the search icon. Find nginx-ingress-controller and click Edit in the Actions column.
In the upper part of the Edit page, select the nginx-ingress-controller container. On the Environments tab, click Add to add the following environment variable.
Type
Variable Key
Value/ValueFrom
Description
Custom
OTEL_EXPORTER_OTLP_HEADERS
authentication=<Authentication token>Specify the authentication token you obtained in Step 1: Obtain an endpoint from Managed Service for OpenTelemetry. Example:
authentication=bfXXXXXXXe@7bXXXXXXX1_bXXXXXe@XXXXXXX1.
After you add the environment variable, click Update on the right side of the Edit page. In the message that appears, click Confirm.
In the left-side navigation pane, choose Configurations > ConfigMaps.
In the upper part of the ConfigMap page, select kube-system from the Namespace drop-down list. Then, enter nginx-configuration in the Name search box and click the search icon. Find nginx-configuration and click Edit in the Actions column.
In the Edit panel, click Add to add the following configurations and click OK.
Name
Description
Value
enable-opentelemetry
Specifies whether to enable Managed Service for OpenTelemetry.
true: enables Managed Service for OpenTelemetry.main-snippet
-
env OTEL_EXPORTER_OTLP_HEADERS;otel-service-name
Enter a custom service name.
Example: nginx-ingress.
otlp-collector-host
The domain name used to import data over gRPC.
Delete
http://and the port number from the VPC endpoint you obtained in Step 1: Obtain an endpoint from Managed Service for OpenTelemetry. Example: tracing-analysis-XX-XX-XXXXX.aliyuncs.com.otlp-collector-port
The port used to import data over gRPC.
8090opentelemetry-trust-incoming-span
Specifies whether to trust the call traces of other services or systems.
Set a value of
trueto trust the call traces of other services or systems.opentelemetry-operation-name
The span format of call traces.
HTTP $request_method $service_name $uri
Step 3: View the trace data in the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console
In the left-side navigation pane, click Applications.
In the upper part of the Applications page, select the region from which you want to import trace data. Then, click nginx-ingress.
On the application details page, you can view the trace data.
On the Application Overview tab, you can view the number of requests and the number of errors.
On the Trace Analysis tab, you can view the trace list and average duration.
On the Trace Analysis tab, you can click a trace ID to view the trace details.
OpenTracing
Step 1: Obtain an endpoint from Managed Service for OpenTelemetry
New version of the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console. In the left-side navigation pane, click Integration Center.
In the Open Source Frameworks section, click the Zipkin card.
NoteObtain an endpoint of the client used to collect data. In this example, a Zipkin client is used.
In the Zipkin panel, select the region from which you want to import trace data.
Record the endpoint.
NoteThe NGINX Ingress controller is deployed on Alibaba Cloud and resides in the same region as the Managed Service for OpenTelemetry agent in this example. Therefore, we recommend that you use a VPC endpoint. If the NGINX Ingress controller and the Managed Service for OpenTelemetry agent are deployed in different regions, use a public endpoint.
Previous version of the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console.
In the left-side navigation pane, click Cluster Configurations. On the page that appears, click the Access point information tab.
In the upper part of the page, select the region from which you want to import trace data.
Select Show Token in the Cluster Information section and click Zipkin in the Client section. Then, record the endpoint.
NoteThe NGINX Ingress controller is deployed on Alibaba Cloud and resides in the same region as the Managed Service for OpenTelemetry agent in this example. Therefore, we recommend that you use a VPC endpoint. If the NGINX Ingress controller and the Managed Service for OpenTelemetry agent are deployed in different regions, use a public endpoint.
Step 2: Enable Managed Service for OpenTelemetry for the NGINX Ingress controller
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose .
In the upper part of the ConfigMap page, select kube-system from the Namespace drop-down list. Then, enter nginx-configuration in the Name search box and click the search icon. Find nginx-configuration and click Edit in the Actions column.
Configure Zipkin as the client to collect trace data.
In the Edit panel, click Add. Enter zipkin-collector-host in the Name field and enter the endpoint you obtained in Step 1: Obtain an endpoint from Managed Service for OpenTelemetry in the Value field.
ImportantRemove
http://from the endpoint and add a question mark (?) at the end of the endpoint. Example:Original endpoint:
http://tracing-analysis-dc-hz-internal.aliyuncs.com/adapt_******_******/api/v1/spansModified endpoint:
tracing-analysis-dc-hz-internal.aliyuncs.com/adapt_******_******/api/v1/spans?
Enable tracing.
Click Add. Enter enable-opentracing in the Name field and true in the Value field. Then, click OK.
Step 3: View the trace data in the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console.
In the left-side navigation pane, click Applications.
In the upper part of the Applications page, select the region from which you want to import trace data. Then, click nginx.
In the left-side navigation pane of the details page, click Interface Calls. On the right side of the page, you can view the trace data.
On the Overview tab, you can view the trace topology.
Click the Traces tab to view the top 100 time-consuming traces of the application. For more information, see Interface calls.
On the Traces tab, you can click a trace ID to view the trace details.
(Optional) Change the protocol used by Managed Service for OpenTelemetry to pass trace data
If you use Managed Service for OpenTelemetry to enable tracing for the NGINX Ingress controller, Managed Service for OpenTelemetry passes trace data in the W3C trace context specification to the downstream service. If the frontend and backend applications use other protocols, such as Jaeger or Zipkin, you must change the protocol used by Managed Service for OpenTelemetry to pass trace data. This way, the traces of the frontend application, NGINX Ingress, and backend application can be streamlined. The following example describes how to change the protocol used by Managed Service for OpenTelemetry to pass trace data:
Add the OTEL_PROPAGATORS environment variable in Step 2.4: Add the OTEL_EXPORTER_OTLP_HEADERS environment variable. Then, save the changes and redeploy nginx-ingress-controller.
Variable key
Value
Description
OTEL_PROPAGATORS
tracecontext,baggage,b3,jaeger
The protocol used to pass trace data. For more information, see Specify the format to pass trace data.
Modify the main-snippet configurations in Step 2.7: Modify the nginx-configuration ConfigMap to make the OTEL_PROPAGATORS environment variable take effect.
Name
Value
Description
main-snippet
env OTEL_EXPORTER_OTLP_HEADERS; env OTEL_PROPAGATORS
Load the OTEL_PROPAGATORS environment variable.
References
For more information about Managed Service for OpenTelemetry, see What is Managed Service for OpenTelemetry?
For more information about ACK, see What is ACK?
To use different clients to collect trace data, such as Zipkin, Jaeger, and SkyWalking, you must configure different parameters in the nginx-configuration ConfigMap. For more information about how to import data to Managed Service for OpenTelemetry, see Preparations.