You can enable tracing for NGINX Ingress Controller deployed in a Container Service for Kubernetes (ACK) cluster, and import trace data to Managed Service for OpenTelemetry. Then, you can view traces and topology in Managed Service for OpenTelemetry to troubleshoot issues related to NGINX Ingress Controller. This topic describes how to enable tracing for NGINX Ingress Controller.
Prerequisites
The NGINX Ingress controller is installed. For more information, see Manage the NGINX Ingress controller.
Version description
The support for tracing varies based on the NGINX Ingress controller version. The following table describes the NGINX Ingress controller versions that support and do not support tracing.
NGINX Ingress controller version | OpenTelemetry | OpenTracing |
≥ 1.10.2-aliyun.1 | Supported | Not supported |
v1.9.3-aliyun.1 | Supported | Supported |
v1.8.2-aliyun.1 | Supported | Supported |
< v1.8.2-aliyun.1 | Not supported | Supported |
Procedure
Managed Service for OpenTelemetry
Step 1: Obtain the endpoint information
Log on to the ARMS console. In the left-side navigation pane, click Integration Center.
On the Integration Center page, click the OpenTelemetry card in the Server-side Applications section.
In the OpenTelemetry panel, select the region from which you want to report data.
Record the endpoint.
NoteBecause NGINX Ingress Controller is deployed in the same region as the Managed Service for OpenTelemetry agent in this example, we recommend that you use a virtual private cloud (VPC) endpoint. If NGINX Ingress Controller and the agent are deployed in different regions, use a public endpoint.
Step 2: Enable Managed Service for OpenTelemetry for NGINX Ingress Controller
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, choose .
In the upper part of the page, select kube-system from the Namespace drop-down list. Then, enter nginx-ingress-controller in the search box and click the search icon. Find NGINX Ingress Controller and click Edit in the Actions column.
In the upper part of the Edit page, select the nginx-ingress-controller container. On the Environments tab, click Add to add the following environment variable.
Type
Variable Key
Value/ValueFrom
Description
Custom
OTEL_EXPORTER_OTLP_HEADERS
authentication=<Authentication token>
Use the authentication token obtained in Step 1: Obtain the endpoint information. Example:
authentication=bfXXXXXXXe@7bXXXXXXX1_bXXXXXe@XXXXXXX1
.After you add the environment variable, click Update on the right side of the Edit page. In the message that appears, click Confirm.
In the left-side navigation pane, choose Configurations > ConfigMaps.
In the upper part of the ConfigMap page, select kube-system from the Namespace drop-down list. Then, enter nginx-configuration in the Name search box and click the search icon. Find nginx-configuration and click Edit in the Actions column.
In the Edit panel, click Add to add the following configurations and click OK.
Item
Description
Value
enable-opentelemetry
Specifies whether to enable Managed Service for OpenTelemetry.
true
: enables Managed Service for OpenTelemetry.main-snippet
-
env OTEL_EXPORTER_OTLP_HEADERS;
otel-service-name
The custom service name.
Example: nginx-ingress.
otlp-collector-host
The domain name used to import data over gRPC.
Delete
http://
and the port number from the VPC endpoint obtained in Step 1: Obtain the endpoint information. Example: tracing-analysis-XX-XX-XXXXX.aliyuncs.com.otlp-collector-port
The port used to import data over gRPC.
8090
opentelemetry-trust-incoming-span
Specifies whether to trust the traces of other services or systems.
Set a value of
true
to trust the traces of other services or systems.opentelemetry-operation-name
The format of spans.
HTTP $request_method $service_name $uri
Step 3: View the trace data in the ARMS console
Log on to the ARMS console. In the left-side navigation pane, choose .
In the top navigation bar of the Applications page, select a region and click the nginx-ingress application.
On the application details page, view the trace data.
On the Overview tab, view the number of requests and the number of errors.
On the Trace Explorer tab, view the trace list and average duration.
On the Trace Explorer tab, click a trace ID to view the trace details.
OpenTracing
Step 1: Obtain the endpoint information
Log on to the ARMS console. In the left-side navigation pane, click Integration Center.
On the Integration Center page, click the Zipkin card in the Server-side Applications section.
NoteObtain an endpoint of the client used to collect data. In this example, a Zipkin client is used.
In the Zipkin panel, select the region from which you want to report data.
Record the endpoint.
NoteBecause NGINX Ingress Controller is deployed in the same region as the Managed Service for OpenTelemetry agent in this example, we recommend that you use a virtual private cloud (VPC) endpoint. If NGINX Ingress Controller and the agent are deployed in different regions, use a public endpoint.
Step 2: Enable Managed Service for OpenTelemetry for NGINX Ingress Controller
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose .
In the upper part of the ConfigMap page, select kube-system from the Namespace drop-down list. Then, enter nginx-configuration in the Name search box and click the search icon. Find nginx-configuration and click Edit in the Actions column.
Configure Zipkin as the client to collect trace data.
In the Edit panel, click Add. Enter zipkin-collector-host in the Name field and enter the endpoint obtained in Step 1: Obtain the endpoint information in the Value field.
ImportantRemove
http://
from the endpoint and add a question mark (?) at the end of the endpoint. Example:Original endpoint:
http://tracing-analysis-dc-hz-internal.aliyuncs.com/adapt_******_******/api/v1/spans
Modified endpoint:
tracing-analysis-dc-hz-internal.aliyuncs.com/adapt_******_******/api/v1/spans?
Enable tracing.
Click Add. Enter enable-opentracing in the Name field and true in the Value field. Then, click OK.
Step 3: View the trace data in the ARMS console
Log on to the ARMS console. In the left-side navigation pane, choose .
In the top navigation bar of the Applications page, select a region and click the nginx application.
In the left-side navigation pane of the details page, click Interface Calls. On the right side of the page, view the trace data.
On the Overview tab, view the trace topology.
Click the Traces tab to view the top 100 time-consuming traces of the application. For more information, see Interface calls.
On the Traces tab, click a trace ID to view the trace details.
(Optional) Change the protocol used by Managed Service for OpenTelemetry to pass trace data
If you use Managed Service for OpenTelemetry to enable tracing for the NGINX Ingress controller, Managed Service for OpenTelemetry passes trace data in the W3C trace context specification to the downstream service. If the frontend and backend applications use other protocols, such as Jaeger or Zipkin, you must change the protocol used by Managed Service for OpenTelemetry to pass trace data. This way, the traces of the frontend application, NGINX Ingress, and backend application can be streamlined. The following example describes how to change the protocol used by Managed Service for OpenTelemetry to pass trace data:
Add the OTEL_PROPAGATORS environment variable in Step 2.4: Add the OTEL_EXPORTER_OTLP_HEADERS environment variable. Then, save the changes and redeploy nginx-ingress-controller.
Environment variable
Value
Description
OTEL_PROPAGATORS
tracecontext,baggage,b3,jaeger
The protocol used to pass trace data. For more information, see Specify the format to pass trace data.
Modify the main-snippet configurations in Step 2.7: Modify the nginx-configuration ConfigMap to make the OTEL_PROPAGATORS environment variable take effect.
Configuration name
Value
Description
main-snippet
env OTEL_EXPORTER_OTLP_HEADERS; env OTEL_PROPAGATORS
Load the OTEL_PROPAGATORS environment variable.
References
For more information about Managed Service for OpenTelemetry, see What is Managed Service for OpenTelemetry?
For more information about ACK, see What is ACK?
To use different clients to collect trace data, such as Zipkin, Jaeger, and SkyWalking, you must configure different parameters in the nginx-configuration ConfigMap. For more information about how to import data to Managed Service for OpenTelemetry, see Preparations.