In this tutorial we will learn more of the jcmd diagnostic command utility which is available since Java 8. We will see how we can gather some useful diagnostic information of a running instance of WildFly.
Currently we have already several diagnostic tools for the JVM like jmap, jstack, jstat. As you will see in this tutorial it's enough to learn just one: jcmd which allows to perform most diagnostic commands just using a single tool. Please note that jcmd must be however used on the same machine where the JVM is running.
The shell jcmd is available into the $JAVA_HOME/bin folder. If you execute it without any parameter it will dump the list of Java Processes which are running:
$ jcmd 6328 sun.tools.jcmd.JCmd 28430 /home/jboss/wildfly-11.0.0.Final/jboss-modules.jar -mp /home/jboss/wildfly-11.0.0.Final/modules org.jboss.as.standalone -Djboss.home.dir=/home/jboss/wildfly-11.0.0.Final -Djboss.server.base.dir=/home/jboss/wildfly-11.0.0.Final/standalone
In order to use jcmd you need to execute some commands against the Java Process (you can reference it either by PID or by its name). The list of available commands depend on the actual JDK you are using. The "help" command however will display which commands are built-in:
$ jcmd 28430 help 28430: The following commands are available: JFR.stop JFR.start JFR.dump JFR.check VM.native_memory VM.check_commercial_features VM.unlock_commercial_features ManagementAgent.stop ManagementAgent.start_local ManagementAgent.start GC.rotate_log Thread.print GC.class_stats GC.class_histogram GC.heap_dump GC.run_finalization GC.run VM.uptime VM.flags VM.system_properties VM.command_line VM.version help
As you can see, there's plenty of useful commands available in the current JDK I'm using (OpenJDK 9). Let's see for example how to gather a Thread dump of a Java process:
$ jcmd 28430 Thread.print 28430: 2018-01-22 09:25:00 Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode): "Attach Listener" #178 daemon prio=9 os_prio=0 tid=0x0000000003755000 nid=0x1538 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE . . . . .
How to print all the system properties set for a VM
$ jcmd 28430 VM.system_properties jboss.qualified.host.name=localhost.localdomain jboss.host.name=localhost javax.xml.transform.TransformerFactory=__redirected.__TransformerFactory jboss.server.name=localhost . . . . .
Print all the flags used for a VM (This is quite useful to gather quickly the initial and maximum heap size)
$ jcmd 28430 VM.flags 28430: -XX:CICompilerCount=3 -XX:InitialHeapSize=67108864 -XX:MaxHeapSize=536870912 -XX:MaxMetaspaceSize=268435456 -XX:MaxNewSize=178782208 -XX:MetaspaceSize=100663296 -XX:MinHeapDeltaBytes=524288 -XX:NewSize=22020096 -XX:OldSize=45088768 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseFastUnorderedTimeStamps -XX:+UseParallelGC
You can check what is the Java process uptime in seconds
$ jcmd 28430 VM.uptime 28430: 28416.487 s
Another useful feature is creating a Class histogram, which provides at your finger tips how many instances of a Class is around. This command can be a bit verbose so you can redirect it to a file or use a grep to filter just on some classes:
$ jcmd 28430 GC.class_histogram | grep "java.lang.Thread" 59: 75 28200 java.lang.Thread 89: 553 17696 java.lang.ThreadLocal$ThreadLocalMap$Entry 103: 190 15712 [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry; 215: 190 4560 java.lang.ThreadLocal$ThreadLocalMap 386: 5 1152 [Ljava.lang.Thread; 454: 17 816 java.lang.ThreadGroup 455: 51 816 java.lang.ThreadLocal 695: 14 336 java.lang.ThreadLocal$SuppliedThreadLocal 1074: 3 160 [Ljava.lang.ThreadGroup; 1136: 6 144 java.lang.Thread$State 1592: 2 80 [Ljava.lang.Thread$State;
You can create as well an heap dump (hprof dump) with jcmd, by providing the filename:
$ jcmd 28430 GC.heap_dump filename=Myheapdump 28430: Heap dump file created
You can run a full Garbage Collector cycle as well:
$ jcmd 28430 GC.run 28430: Command executed successfully
Additionally, jcmd can be used to produce a Java Flight Recorder diagnostic output. The Java Flight Recorder is a great tool to investigate performance issues, however it is a commercial tool from Oracle JDK, therefore check its license agreements: https://docs.oracle.com/javase/9/troubleshoot/troubleshoot-performance-issues-using-jfr.htm
Please note that these commands are available only if the Java application was started with the Java Flight Recorder enabled, that is, using the following options: -XX:+UnlockCommercialFeatures -XX:+FlightRecorder
Let's see how we can start a recording of a JFR session:
Here is how to start a 10-minute recording on the running Java process with the identifier 28430 and save it to session1.jfr in the current directory, use the following:
$ jcmd 28430 JFR.start name=Session1 duration=10m filename=session1.jfr
To check the status of all recordings running for the specified process, including the recording identification number, file name, duration use the JFR.check diagnostic command:
$ jcmd 28430 JFR.check
The JFR.stop diagnostic command stops a running recording and has the option to discard the recording data. For example:
$ jcmd 28430 JFR.stop
Finally, you can dump the data collected so far by the recording with a specific identification number
$ jcmd 28430 JFR.dump