Mac and Memory
There was a grad student here that wrote a Perl program to compute the Jaccard coefficent of some Blast data. The Mac he was using has 8G of memory, but come to about 2GB of memory usage Perl would crash. I reworte it into two programs, first to dump all his data to a gdbm file, then processed that file using the Jaccard coefficent. It never used more then 2G and finished, it took like a day longer to run but we got what we wanted. This all happened a few months ago, and for this and other reasons we moved that project to Solaris.
A few days ago one of our sysadmin was investigating a complaint on why Perl on MacOS X was slow. He built the same Perl twice, one using the system’s malloc(3), the other using the malloc shipped with Perl. Using the original program computing Jaccard coefficent, with a small dataset, it used 1,220 CPU seconds with Apple’s malloc, but only 605 CPU seconds with Perl’s! I asked him to run it with the full data set. Apple’s malloc crashed after 116,700 CPU seconds and ~2.3GB of memory. With Perl’s malloc it used 65,469 CPU seconds and finished after using ~2.8GB of memory.
So happy SysAdminDay!
