Application Real-Time Monitoring Service:Customize settings for a Java application

Prerequisites

An ARMS agent is installed for the application. For more information, see Application Monitoring overview.

Feature entry

1. Log on to the ARMS Console, and in the left-side navigation pane, select Application Monitoring > Application List.

2. At the top of the Application List page, select the destination region, and then click the target application name.

Note

The icon meanings in the Language column are as follows:

: Java application with application monitoring access.

: Golang application with application monitoring access.

: Python application with application monitoring access.

-: Application with Managed Service for OpenTelemetry access.

3. In the top navigation bar, select Application Configuration > Custom Configuration.

4. Set the custom configuration parameters. After setting, click Save.

Sampling settings

In the Sampling Settings area, you can set the sampling strategy and sampling interface name for the trace. For more details, see Trace Sampling Mode Selection (Agent Version 3.2.8 and Above).

Agent switch settings

In the Agent Switch Settings area, you can control the start and stop of application monitoring and adjust each plug-in switch.

Thread profiling settings

In the Thread Profiling Settings area, you can turn on or off the main switch for thread profiling, the switch for periodically saving local method stack information, and set the threshold for high response time calls.

Application log association configuration

In the Application Log Association Configuration area, you can set the log source information associated with the application and automatically fill in TraceId and SpanId. For more information, see Log Analysis.

• Auto Fill TraceId and SpanId Configuration Effect: After enabling, Trace ID and Span ID can be automatically injected into logs without manually modifying the log configuration file.

  Configuration effect:

  As shown in the following figure, log4j, log4j2, and logback automatically print Trace ID and Span ID in logs without modifying the log configuration file.

  

• Add traceId in log MDC: By default, the 4.x agent no longer supports obtaining TraceId through org.slf4j.MDC.get(“EagleEye-TraceID”). If you still need to obtain it in this way, you can enable the current switch.

URL convergence settings

In the URL Convergence Settings area, you can turn on or off the convergence function switch and set the convergence threshold and convergence rules. URL convergence refers to displaying a series of similar URLs as a single entity. For example, URLs prefixed with /service/demo/ are displayed as an object. The convergence threshold is the minimum number of URLs that can trigger URL convergence. For example, if the threshold is set to 100, URLs are converged only when 100 URLs meet the regular expression of the rule. For more information, see ARMS Convergence Mechanism Description.

Arthas monitoring

In the Arthas Monitoring area, you can turn on or off the Arthas diagnostic function and set the effective IP. For more information, see Arthas Diagnostics.

Continuous profiling settings

In the Continuous Profiling Settings area, you can turn on or off the main switch, CPU hotspot, memory hotspot, and code hotspot functions, and set the effective IP or CIDR block. For more information, see Java Application Continuous Profiling.

Data masking settings

In the Data Masking Settings area, by setting masking rules, the agent will mask sensitive data such as JVM system parameters, K8s Yaml, method input parameters, Arthas environment variables, and system variables during data collection. Elements in the masking rules are separated by commas, and each element represents a case-insensitive regular expression. For example, the masking rule for password is equivalent to the regular expression .*password.*.

When the ARMS agent collects data, the agent masks the data based on the key. If a key matches a regular expression, the corresponding value is masked. As shown in the following figure, when the masking rule is licenseKey, the information with -Darms.licenseKey as the key is masked.

Interface call configuration

In the Interface Call Configuration area, you can set strategies such as slow call threshold, exception whitelist, HTTP status code whitelist, and invalid interface call filtering.

• Slow call threshold: The default is 500. When the interface response time exceeds this threshold, the interface is marked as a slow call.

• HTTP status code whitelist:

  By default, HTTP status codes greater than 400 are classified as error calls. If you do not want a certain type of status code to be classified as an error, you can set a whitelist to ignore such errors.

  This only affects the HTTP frameworks currently supported by application monitoring.

  Impact on data: Error count metrics for HTTP server/client (arms_http_requests_error_count, arms_http_client_requests_error_count, arms_app_requests_error_count), Span status.

  Impact on features: Error counts in the Application Overview, Provided Services, and Dependent Services tabs, Span status, and error count alerts on the Trace Analysis page.

  Content format: Fill in a single status code, separate multiple status codes with commas, and fuzzy matching is not supported.

  Example: 403,502

  Default value: Empty

• Invalid interface call filtering:

  If you do not want to see such calls on the Provided Services tab, you can enter the interface names that you do not need to view, thereby hiding them from the interface call page.

  Note

  • For Java agents earlier than version 4.2.0, this feature is only effective for interfaces appearing in Provided Services.

  • For Java agents version 4.2.0 and above, this feature is effective for any LocalRootSpan.

  Impact on data: All metrics and Spans corresponding to the interface will be ignored.

  Impact on features: All metrics corresponding to the interface on the Application Overview, Provided Services, and Dependent Services tabs, Span count, call volume, error count, and slow call alerts on the Trace Analysis page.

  Content format: Use string or AntPath expression to match the full name of invalid interfaces. Separate multiple rules with commas. (The default AntPath expression is written for compatibility with historical data. It is not recommended to delete it. Please append new configurations to the existing rules).

  Example: /api/test/*,/api/playground/create

  Default value: /**/*.jpg,/**/*.png,/**/*.js,/**/*.jpeg,/**/*.pdf,/**/*.xlsx,/**/*.txt,/**/*.docs,/**/*.gif,/**/*.csv

• Whether to record upstream interface name in interface call metrics and Whether to record upstream application name in interface call metrics:

  Configuration effect: Controls whether the upstream application and upstream interface that called the interface are recorded in the interface metrics, mainly affecting whether there is data in the upstream and downstream links in the provided services. When an application has many upstream applications, recording this information may lead to a significant increase in the amount of metric reports, thereby increasing costs.

  Configuration effect: As shown in the figure, after disabling the recording of upstream applications and interfaces in interface metrics, the corresponding data drops to 0.

  

• Whether to record original status code in interface call metrics:

  Configuration effect: Records the original response code in HTTP interface-related metrics.

  Configuration effect:

  

Database call configuration

In the Database Call Configuration area, you can set the slow SQL threshold, the maximum retention length for collected SQL, and whether to display variable binding values and constant values in SQL, and whether to record the size of MySQL query return values.

• Display variable binding values in SQL: Captures variable values bound by PrepareStatement parameters, which take effect without restarting the application.

• Display constant values in SQL: Only truncates SQL without additional processing, which takes effect without restarting the application.

Trace propagation protocol settings

In the Trace Propagation Protocol Settings area, you can select the Trace protocol to use according to your needs. For the Trace protocols supported by ARMS, see Tracing Protocols Supported by ARMS.

By default, when a call arrives, the ARMS agent will sequentially check for the presence of request headers defined by the protocols in the order of EagleEye, OpenTelemetry, SkyWalking, Jaeger, and Zipkin. If a protocol is detected, the trace context will be restored according to the protocol specifications. When making subsequent calls to downstream services, related headers will be inserted into the requests as per the specifications. If none of the aforementioned detections succeed, the EagleEye protocol will be used by default.

You can select any one protocol as the preferred protocol on this page. After selecting and saving your preference, ARMS will prioritize checking for the presence of request headers specified by the selected protocol. For example, with the following configuration, when a call arrives, the ARMS agent checks for the protocol header in the order of Jaeger, EagleEye, OpenTelemetry, SkyWalking, and Zipkin.

You may also choose to enforce the use of a specific protocol. For example, with the following configuration, when a call arrives, the ARMS agent will only check for the presence of request headers defined by the Jaeger protocol. If these headers are not detected, it will not proceed to check for other protocols; instead, it will generate a new trace context.

Configuration effect:

Supports selecting a preferred or enforced tracing propagation protocol. There are generally the following usage scenarios:

• Requests carry trace contexts in multiple formats, and the default parsing order does not meet requirements. You want to specify a preferred protocol for parsing. For example, if requests carry both W3C and Zipkin trace contexts, by default, the system prioritizes parsing the W3C trace context. You can change this behavior by setting the propagation protocol to Zipkin, thereby prioritizing the Zipkin protocol instead.

• Requests carry trace headers of a specific protocol, but you do not want to reuse those headers and prefer to generate a new trace context using another protocol. For instance, if requests carry Zipkin trace contexts, by default, the system parses the Zipkin trace context. By setting the propagation protocol to W3C and changing the propagation mode to enforced, the system will ignore the Zipkin trace context and generate a new context according to the W3C protocol specifications. Note that if you do not change the propagation mode to enforced, the system will still attempt to parse the Zipkin trace context after failing to parse the W3C context.

Configuration effect:

After configuration, each LocalRootSpan in a trace will include an attribute with the key trace.protocol.type and the value as the current trace protocol used in the trace.

Message queue configuration

In the Message Queue Configuration area, you can customize consumer information.

• Customize RabbitMQ Consumers: Specify the class name for your custom consumer or the class containing the anonymous inner consumer to view the consumer's call link. Separate multiple consumers with commas.

• Customize Kafka Consumption Methods: Customize your consumption method to view link and metric data when consuming messages using the native Kafka SDK in your scenario.

• Automatic Context Propagation for Kafka Messages: Adds headers to Kafka messages automatically to link the sending and consumption processes.

Agent collection configuration

In the Agent Collection Configuration area, you can set the maximum trace collection rate per second, the maximum processable QPS threshold, and the agent log level.

• Whether to collect trace data: Controls whether trace data is reported. It is enabled by default. After disabling, trace data will no longer be reported.

• Maximum trace collection rate per second: The number of Spans that the agent can report per second (for performance considerations, the actual effective threshold and user-configured threshold have a deviation of less than 5%). Spans exceeding this part will not be reported.

• Maximum processable QPS threshold for the agent: The number of requests that the agent can process per second (for performance considerations, the actual effective threshold and user-configured threshold have a deviation of less than 5%). Requests exceeding this threshold will not be monitored, that is, Spans or metrics will not be generated for requests exceeding the threshold, and the log association TraceId function will not take effect.

• Collect data for internal calls without entry points: In ARMS, HTTP services, RPC services, scheduled tasks triggered by applications, and message consumption provided by applications are considered entry points. Calls such as database calls and HTTP requests that occur in the business logic of these entry points are considered internal calls with entry points. Conversely, HTTP calls, database calls, NoSQL calls, message sends, and RPC calls initiated via JDK thread pools are considered internal calls without entry points. The ARMS agent V4.2.2 and later supports filtering out such data with one click. As shown in the figure below, in the Lettuce framework, commands are periodically executed automatically to ensure a normal connection with the Redis server. Such calls fall under internal calls without entry points. After disabling the collection of such data at the time point shown in the figure, the related data disappears.

  

• Agent log level: Adjusts the log level of the agent for troubleshooting.

Advanced exception filtering configuration

In the Advanced Exception Filtering Configuration area, you can set exception collection rules.

• Collect plug-in exceptions: Whether to collect plug-in exceptions.

• Stack depth for distinguishing similar exceptions: The default is 2. Based on this stack depth, the system identifies the same type of exceptions. Modifying this configuration may cause unexpected statistical behavior. Proceed with caution.

• Exception filtering whitelist:

  If you do not want to see such exceptions on the Exception Analysis tab, you can enter the fully qualified class name of the exception that you do not want to be counted as an exception, thereby hiding it from the exception analysis page.

  Impact on data: Exception count metrics (arms_exception_requests_count_raw, arms_exception_requests_seconds_raw), exception information in Span.

  Impact on features: Metrics corresponding to exceptions on the Exception Analysis tab, exception information in Span on the Trace Analysis page, and exception count alerts for corresponding exceptions.

  Content format: Use the fully qualified class name of the exception to identify the exceptions that need to be filtered. Separate multiple exceptions with commas.

  Example: java.lang.InterruptedException,java.lang.IndexOutOfBoundsException

  Default value: None

• Inheritance of exception filtering parent class: After enabling, if the currently collected exception is a subclass of the exception class configured in the Exception Filtering Whitelist, it will also be filtered.

  Configuration effect: Exceptions that meet the filtering criteria will not be displayed in the ARMS console.

• Exception message filtering: After configuration, the message field of the specified type of exception that meets the configuration conditions will also be filtered.

  • Exception name: Specifies which exception the filter applies to.

  • Message conditions: The conditions of the exception message, which can be startsWith, endWith, or contains.

  • Message keywords: The keyword string.

  Configuration effect: Exceptions that meet the filtering criteria will not be displayed in the ARMS console.

Pooling monitoring configuration

In the Pooling Monitoring Configuration area, you can set collection rules related to thread pools and connection pools.

• Thread pool and connection pool monitoring: Supports monitoring of thread pool metrics for frameworks such as Tomcat, Dubbo, and HSF. You need to upgrade the agent to the latest version.

• Thread pool thread name pattern extraction strategy: By default, this feature replaces all numeric characters in the thread name of any running thread in the thread pool with *. You can also adjust it to replace only the ending characters of the thread name with *. In applications where multiple Dubbo Providers are initiated and these providers have different listening ports, if the default strategy is applied, two thread pools from two Dubbo Providers could be aggregated into one due to identical extracted thread name templates. At this point, adjusting this strategy can help distinguish between the two thread pools.

  Configuration effect: Taking the common Tomcat thread pool as an example, the default displayed thread pool thread name is http-nio-*-exec-*. After adjustment, the thread name is http-nio-9099-exec-*.

  

• Thread pool usage scenario filtering and Thread pool thread name pattern filtering: Filters out the monitoring metrics of certain thread pools based on the thread pool usage scenario and thread pool thread name pattern.

  Note

  This configuration only takes effect for Java agents version 4.2.0 and above.

  • Thread pool usage scenario: Refers to the context in which the thread is used, currently supporting several types including Tomcat, Vert.x, Undertow, Dubbo, Jetty, AliyunJavaAgent, and default. Among these, AliyunJavaAgent represents the thread pool used by the agent, while "default" stands for other unclassified thread pools.

  • Thread pool thread name pattern: Refers to the thread name pattern obtained after processing the thread name in the thread pool. For example, http-nio-*-exec-* is generally obtained by replacing the numeric part in the actual thread with *.

  Impact on data: Thread Pool Metrics

  Impact on features: Thread Pool Monitoring tab, alerts configured based on thread pool monitoring metrics.

  Content format:

  • Thread pool usage scenario filtering: The usage scenarios displayed on the thread pool monitoring tab. Separate multiple usage scenarios with commas.

  • Thread pool thread name pattern filtering: The thread name patterns displayed on the thread pool monitoring tab. Separate multiple usage scenarios with commas. The matching method is exact match, and rule matching is not supported.

  Example:

  • Thread pool usage scenario filtering: AliyunJavaAgent,Jetty

  • Thread pool thread name pattern filtering: Catalina-utility-*,DubboServerHandler-*-thread-*

  Represents that all thread pool data in the AliyunJavaAgent scenario is not reported. In other scenarios, if the thread pool pattern name is Catalina-utility-*, it is also not reported.

  Default value: None

Span attributes configuration

• Record OTel Spec specified attributes: The OpenTelemetry Specification specifies the attributes that should be included in the Span generated for each plug-in type. However, due to data reporting considerations, the ARMS agent does not record these attributes in the Span by default. You can enable it according to your needs. After enabling, see the OpenTelemetry Specification for the attributes that each framework will add.

  The following example shows a span for an HTTP server, where the attributes within the red box are the newly added attributes after enabling the switch.

  

• Span association application tag configuration: Usedto control which Spans the tags bound to the application on the Application List page in the console are attached to. By default, all Spans will contain application tags. Considering usage volume, you can attach application tags only to entry spans, such as spans of HTTP servers, RPC servers, message queues, or scheduled tasks.

  After configuring to attach to entry spans, the effect is as shown in the following figure. For example, the application contains an application tag pair such as test1:value1.

  

  For entry spans, you can see the corresponding tag attributes.

  

  For non-entry spans, the attributes test1:value are not included.

  

Advanced settings

In the Advanced Settings area, you can set the interfaces to be filtered, the maximum length of the method stack, and more.

• Quantile statistics: Whether to enable the quantile statistics function.

• Maximum retention length of method stack: The default is 128 entries, and the maximum is 400 entries.

• Trace compression: Whether to simplify repeated calls (such as for loops).

• Maximum display length of request input parameters: The default is 512 characters, and the maximum supported length is 2048 characters.

• Asynchronous propagation scan package name: Add an asynchronous propagation scan package to implement asynchronous task monitoring. After a Runnable object, a Callable object, or a Supplier object is created, the corresponding method in the scan package for asynchronous propagation automatically captures the trace context of the current thread. Then, when threads are used in asynchronous mode, the method propagates the captured trace context to the threads.

• Include TraceId in HTTP response results: Only for HTTP-type requests, returns the field eagleeye-traceid in the Response Header.

• Encrypted reporting: After enabling, ARMS will report data through TLS encryption.

Copy application settings to other applications

If you need to synchronize the same configuration for other applications, you can copy the corresponding configuration to other applications.

Globally apply application settings

You can globally apply the current application settings. If you create a new application, the settings are used by default.

1. Click Save Current Application Settings as Global Default at the bottom of the page.

2. If the Current Settings Not Saved dialog box appears, click OK to save the configuration of this application, and then click Save Current Application Settings as Global Default.

3. Click Confirm in the pop-up dialog box.

FAQ