MemoryLeak Analysis using Java SE 6

If you are having a memory leak you will get OutOfMemoryError (OOM). You may get OOM even if you are not having memory leak if the following cases
a) Huge amount of local variables (short living objects)
b) Large number of concurrent threads running
c) Complex application and insufficient amount of heap size (memory required for long living objects)
You can conclude if there is a memory leak or not by analyzing the jstat report (jstat -gc 30s) or by analyzing the jConsole graph.
If the bottom of the zig zag graph keep increasing there is a memory leak.
If the bottom of the zig zag graph is greater than the available heap then it is condition (c)
If the peak of zig zag graph is greater than the heap then it is condition (a)
OOM could in heap area, permanent generation or in native memory area
If OOM in heap then the message could be "OutOfMemoryError: Java heap space"
If OOM in permanent generation then "OutOfMemoryError: PermGen space"
If OOM in native area then "java.lang.OutOfMemoryError: request bytes for . Out of swap space?"
If you have memory leak in perm gen then verbose:class may help. You need to find out the class which is getting repeatedly loaded without getting unloaded.
For native area, no specific tools available.
If you have OOM in heap then you can use Jmap and jhat to analyze further.
Use the application for a while. Repeat the operations which you suspect causing the memory leak.
Take a heap dump of the JVM by jmap.exe -dump:live,format=b,file=heap1.dmp
Repeat the suspecting operations further
Take the heap dump again. Name the dump file heap2.dmp
Run Jhat over both dumps jhat -J-Xmx512m -baseline heap1.dmp heap2.dmp
In a browser type http://localhost:7000/histo/
From the histogram, find out the classes which is having highest total size. These are suspected classes for memory leak.
Click one class, another page opens where you can see all instances of that class. New Instances are marked new.
Click one instance which is new. Another page opens where you see reference chain from root (use exclude weak ref)
Click that link, which displays the reference from root to that instance, Study the path carefully. You may need do this analysis for couple of classes in the histogram, you will find some references which is otherwise not necessary.
Study the source code remove that references.
References
http://java.sun.com/developer/technicalArticles/J2SE/monitoring/
http://weblogs.java.net/blog/jfarcand/archive/2006/02/using_mustangs.html