Too many open files

I owned a web server about a year ago that would crash every other night. I did everything I could to try to track down the exception (Too many open files), but I couldn't figure out what was causing it. I eventually chalked it up to a glassfish bug since they did have that problem before.

Well today I hit it again, this time in a tomcat instance. I ran across a form posting suggesting I try this command:

ls -l /proc/[pid]/fd

Which is an awesome command that I wish I knew about a year ago. This showed all the files that were opened which were indeed my fault. The problem came from a dom4j reader that wasn't being closed which was leaking the file connection. By closing the datastream the problem went away.

So if you have this problem on linux try the above command and see who is opening that file.


Hope that helps,
Chris