Hi, my first post....first, I'm looking for a place appropriate for this, so if this is not it, please pardon me. Any pointers to an online forum for HPC would be more appreciated.
I have a Dell Tower with 64 gigs of ram, 12 processors, a 250gb SST, and 2 2tb hard drives in RAID 0.
I run matlab, to analyze EEG (brainwave) data sets, about 4gb each. When I run an independent component analysis (ICA), all the cores are engaged, and the ram is about half full, taking about a day to complete. That is for one subject. I also do other analyses which are not multi-threaded, so don't use all 12 processors, but operate on 15 4gb datasets (all 15 subjects in an experiment) at once. The toolbox I use is designed to pull data from the hard drive as needed for this, since few machines will have RAM for 15 4gb datasets. Also, as any of these analyses are done, equally large output files are created.
I have windows 7 and Matlab installed on the SST. I am thinking of reconfiguring the system with Windows 7 and Matlab on the RAID to leave the SST free for the EEG data. My thinking is that the ICA and the 15-subject analyses involve a lot of back and forth between the drive and the CPUs, and to optimize performance here, I need to move the OS and applications off of the SST. At the moment it's pretty full with the OS and applications, leaving not much room for the SST to be used in analyses. I hope having the data on the SST will cause the SST to be the main place where data is read from and written to during the analyses.
Thanks much,
Jim Kroger
NMSU
Unless you have a lot of I/O that is involved in running second application I am not sure if you are going to see a big difference in execution times (I assume that is what you are going for?). And if you do then you may have to consider the following.
How often you need to do second analysis? Consumer grade SSD's are rated for certain read/write "endurance" and (ab)using the SSD drive with a lot of I/O may shorten its life.
I/O is unlikely to be limiting for ICA (or anything else you're doing), which is just matrix manipulations. If you need stuff like this done faster then all you can do is (A) use more cores or (B) use faster processors.
Edit: Note that if you really want to speed things up then using fewer data points will do that.
Thanks guys very much. So you feel I/O is not going to be that much of a factor, whether I am using the SST for reading/writing data to, vs the hard drives. Thank you much!
Jim