First steps for diagnosis of memory leaks in Websphere Application Server v6.1

- Enable verbose GC.

- If using Sun JVM, use the -XX:+HeapDumpOnOutOfMemoryError option to tell the VM to generate a heap dump if OutOfMemoryError is thrown. If using IBM JVM, then enable automatic heap dump generation.

- Start the lightweight memory leak detection

- Generate heap dumps manually if required using the wsadmin tool.

- Use the MDD4J tool (Memory Dump Diagnostic for Java) for diagnosing root causes behind memory leaks in the Java heap. The heap dump collected in the above steps will be the input to this tool. More information about this tool can be found here , here and here.
The MDD4J tool supports the following heap dump formats:
1.IBM Portable Heap Dump (.phd) format (for WebSphere Application Server Versions 6.x on most platforms)
2.IBM Text heap dump format (for WebSphere Application Server Versions 5.0 and 4.0 on most platforms)
3.HPROF heap dump format (for WebSphere Application Server on the Solaris® and HP-UX platforms)
4.SVC Dumps (WebSphere on the IBM zSeries)

On Solaris platform, starting with Java 1.5, Sun has been shipping a cool tool called jmap which allows you to attach to any 1.5 JVM and obtain heap layout information, class histograms and complete heap snapshots. The cool thing is that you don’t have to configure the JVM with any special options, and that it therefore runs exactly as during normal operation.

jmap -dump:format=b,file=snapshot2.jmap PID_OF_PROCESS
jhat snapshot2.jmap