Deploy a data collection environment - Data Transmission Service

This topic describes how to deploy a data collection environment to collect data.

1. Perform a precheck

Make sure that the collector is deployed on a standalone server that has no online applications.
Make sure that JAVA_HOME and JDK 1.6 or later have been configured.

2. Start the collector

If you use a UNIX-based server, run the following command in the directory in which the collector is installed:
 `./run.sh`
 If you use a Windows-based server, run the following command in the directory in which the collector is installed:
 `start /b java -jar javaagent-collector.jar`

View logs in the collector/logs/collector.log file. If the log file contains a startup or success message, the collector is deployed.

For information about how to resolve errors, see Application Collector FAQ.

Java agent

1. Configurations before startup

Make sure that the JAVA_HOME environment variable has been configured. Otherwise, set the value of JAVA_HOME in the attach.sh file to the absolute path of JDK. If you use JRE instead of JDK, you must manually copy the tools.jar file to the ${JAVA_HOME}/lib/ directory.
Configure the javaagent.config file: profiler.collector.ip = 11.23.45.67 # Specify the IP address of the collector. profiler.collector.port = 9996 # Specify the collector port. profiler.app.name = adamApp # Specify the application name. The application name must be less than 20 characters in length and contain only letters and digits. profiler.app.port = 8080 # Specify the startup port of the application. The application may use different ports for different features. You need to configure only the requested port. Configure one port regardless of the number of applications in a JVM. profiler.applicationservertype = TOMCAT # Specify the container type of the application middleware. Valid values: TOMCAT, JBOSS, and WEBLOGIC.

Optional configurations

        Configure prefixes for the directories in which the Java code to be tested resides. Replace the directories in the following example. Separate multiple directories with commas (,). Each directory must have at least two levels of subdirectories. We recommend that you specify no more than five directories.
        If databases are changed, the collector can provide specific suggestions to modify application configurations based on the precise call stack information. 
        If some directories cannot be provided, you can leave this parameter unspecified. You can configure filters on the analysis page of the Alibaba Cloud Advanced Database & Application Migration (ADAM) console. 
        `profiler.classpath.whitelist = com.alibaba.javaagent,com.alibaba.adam`
        If the preceding allowlist is specified, you can leave this parameter unspecified. You can also specify a denylist to filter out unnecessary call stack information. 
        `profiler.classpath.blacklist =org.apache,net.sf`
        Specify the limit of the CPU utilization. When the limit is reached, data collection is paused. 
        `profiler.cpu.threshold = 85`
        Specify the interval at which system information is collected. Unit: minutes. Default value: 15. 
        `profiler.sys.send.interval = 15`
        Specify the interval at which dynamic SQL information is collected. Unit: minutes. Default value: 15. 
        `profiler.sql.dynamic.send.interval = 15`

2. Startup methods

Method 1: Monitor an application only once without restarting the application

Add system operation permissions to the javaagent directory. Make sure that the account used to start the agent is the same as the account used to start the application. Run the following commands, with ${pid} replaced by the process ID of the application:

If you use a UNIX-based server, run the following command:
`./attach.sh -p ${pid}`
If you use a Windows-based server, run the following command:
`java -cp "%JAVA_HOME%\lib\tools.jar;%cd%\javaagent-bootstrap.jar" com.alibaba.adam.javaagent.bootstrap.AgentAttacher -p ${pid}`

If the log file in the javaagent directory contains a success message, the agent has been started. If inbound traffic is received by the application and the collector directory contains a subdirectory that contains data, the agent has been started and has sent data to the collector. For information about how to resolve errors, see Application Collector FAQ.

Make sure that the obtained process ID of the application is correct.
Make sure that the account used to start the agent is the same as the account used to start the application. Make sure that the agent and the application have identical system operation permissions. Otherwise, the agent cannot monitor the application. If the application is started by viewing Windows Registry as a system user, the agent cannot be started by using Method 1 due to inconsistent account permissions.

Method 2: (Recommended) Start monitoring when an application starts

Replace ${javaagent_path} with the directory of the javaagent in the configuration file, and then restart the application.

UNIX operating system

Tomcat: Add the following code to the last CATALINA_OPTS configuration item in the catalina.sh startup file:
        CATALINA_OPTS="$CATALINA_OPTS -javaagent:${javaagent_path}/javaagent-bootstrap.jar"
JBoss: Add the following code to the last JAVA_OPTS configuration item in the run.conf startup file:
        JAVA_OPTS="$JAVA_OPTS -javaagent:${javaagent_path}/javaagent-bootstrap.jar"
WebLogic: Add the following code to the last JAVA_OPTIONS configuration item in the startWebLogic.sh startup file:
        JAVA_OPTIONS="$JAVA_OPTIONS -javaagent:${javaagent_path}/javaagent-bootstrap.jar"
WebSphere:
        Method 1: Add JAVA_OPTS="$JAVA_OPTS -javaagent:/home/admin/javaagent/javaagent-bootstrap.jar" to the configuration file.
        Method 2: Add the following code to the common JVM parameters:
        name:javaagent  value:/home/admin/javaagent/javaagent-bootstrap.jar
Docker containers in Kubernetes clusters:
        Place the javaagent directory into a Docker image, add the corresponding javaagent configuration file, and deploy the image.

Windows operating system

Tomcat: Add the following code to the last CATALINA_OPTS configuration item in the catalina.bat startup file:
        CATALINA_OPTS="$CATALINA_OPTS -javaagent:${javaagent_path}/javaagent-bootstrap.jar"
JBoss: Add the following code to the last JAVA_OPTS configuration item in the run.conf startup file:
        JAVA_OPTS="$JAVA_OPTS -javaagent:${javaagent_path}/javaagent-bootstrap.jar"
WebLogic: Add the following code to the last JAVA_OPTIONS configuration item in the startWebLogic.cmd startup file:
        JAVA_OPTIONS="$JAVA_OPTIONS -javaagent:${javaagent_path}/javaagent-bootstrap.jar"
WebSphere:
        Method 1: Add JAVA_OPTS="$JAVA_OPTS -javaagent:/home/admin/javaagent/javaagent-bootstrap.jar" to the configuration file.
        Method 2: Add the following code to the common JVM parameters:
        name:javaagent  value:/home/admin/javaagent/javaagent-bootstrap.jar

After the application is started, view the application logs. If Java Agent load successfully! is displayed, the agent is started. If an error occurs, modify the script based on the error message. If no agent information is displayed, the agent was not loaded because the configuration path is invalid.

Method comparison

Startup method	Benefit	Drawback	Scenario
Method 1	You can configure the agent without restarting the application even if the application has never been monitored by the agent.	You must manually start the agent each time you restart the application. The account used to start the agent must be the same as the account used to start the application.	This method applies to scenarios in which you will not restart your applications for a long time and an application restart affects your business.
Method 2	When you restart the application, the agent automatically starts and continuously collects application data. You do not need to consider account permissions.	When you collect application data for the first time, you must restart the application to start the agent.	This method applies to scenarios in which you can restart your applications to continuously collect data without affecting your business.

Usage notes

When the application stops, the agent also stops regardless of which method was used to start the agent.
You can stop the agent in the following way.
- If the agent is started by using Method 1, restart the application or run the ./attach.sh -p ${pid} -s command.
- If the agent is started by using Method 2, remove the configurations of the javaagent and restart the application.
If you want to monitor the application after the agent has been stopped, you must restart the application.
- If the agent is started by using Method 1, start the agent manually after the application restarts.
- If the agent is started by using Method 2, wait for the agent to automatically start after the application restarts.
JBoss clusters are divided into Community Edition clusters and Enterprise Edition clusters. If the jboss.modules.system.pkgs configuration item exists, you must add the com.alibaba.adam.javaagent directory and restart the application to validate the monitoring settings regardless whether you use Method 1 or 2.

Collect data (one to seven days of data)

Go to the directory in which the collector is installed, compress the files in the data/ directory in the ZIP format by application name, and separately upload each application profile. Each application corresponds to one profile. You can compress the data collected for an application from different IP addresses into one ZIP file. However, you cannot compress data collected for different applications into one ZIP file. Otherwise, an upload error occurs.

For information about subsequent operations, see Create an application profile.