How to utilise parallel processing in Matlab

22,989

Solution 1

Since you have access to the Parallel toolbox, I suggest that you first check whether you can do it the easy way.

Basically, instead of writing

for i=1:lots
   out(:,i)=do(something);
end

You write

parfor i=1:lots
   out(:,i)=do(something);
end

Then, you use matlabpool to create a number of workers (you can have a maximum of 8 on your local machine with the toolbox, and tons on a remote cluster if you also have a Distributed Computing Server license), and you run the code, and see nice speed gains when your iterations are run by 8 cores instead of one.

Even though the parfor route is the easiest, it may not work right out of the box, since you might do your indexing wrong, or you may be referencing an array in a problematic way etc. Look at the mlint warnings in the editor, read the documentation, and rely on good old trial and error, and you should figure it out reasonably fast. If you have nested loops, it's often best parallelize only the innermost one and ensure it does tons of iterations - this is not only good design, it also reduces the amount of code that could give you trouble.

Note that especially if you run the code on a local machine, you may run into memory issues (which might manifest in really slow execution in parallel mode because you're paging): Every worker gets a copy of the workspace, so if your calculation involves creating a 500MB array, 8 workers will need a total 4GB of RAM - and then you haven't even started counting the RAM of the parent process! In addition, it can be good to only use N-1 cores on your machine, so that there is still one core left for other processes that may run on the computer (such as a mandatory antivirus...).

Solution 2

Mathworks offers its own parallel computing toolbox. If you do not want to purchase that, there a few options

  • You could write your own mex file and use pthreads or OpenMP.
  • However make sure you do not call any Mex api in the parallel part of the code, because they arent thread safe
  • If you want coarser grained parallelism via MPI you can try pmatlab
  • Same with parmatlab

Edit: Adding link Parallel MATLAB with openmp mex files

I have only tried the first.

Solution 3

Don't forget that many Matlab functions are already multithreaded. By careful programming you may be able to take advantage of them -- check the documentation for your version as the Mathworks seem to be increasing the range and number of multithreaded functions with each new release. For example, it seems that 2010a has multithreaded ffts which may be useful for time series processing.

If the intrinsic multithreading is not what you need, then as @srean suggests, the Parallel Computing Toolbox is available. For my money (or rather, my employers' money) it's the way to go, allowing you to program in parallel in Matlab, rather than having to bolt things on. I have to admit, too, that I'm quite impressed by the toolbox and the facilities it offers.

Share:
22,989
Eduardas
Author by

Eduardas

Updated on July 21, 2022

Comments

  • Eduardas
    Eduardas over 1 year

    I am working on a time series based calculation. Each iteration of the calculation is independent. Could anyone share some tips / online primers on using utilising parallel processing in Matlab? How can this be specified inside the actual code?

  • Eduardas
    Eduardas about 13 years
    I have access to the Matlab's Parallel Computing Toolbox. Can you share any good primers / examples on using it?
  • High Performance Mark
    High Performance Mark about 13 years
    @Edward: I attended the Mathworks 2-day training course on the PCT, read the documentation and worked it out for myself. But I have a background in parallel programming with Fortran and MPI and OpenMP. Matlab Central has increasing amounts of stuff on parallel computing with Matlab, that might be a place to start.
  • Amro
    Amro about 13 years
    you might also be interested to know that Parallel Computing Toolbox recently (R2010b) introduced CUDA-based GPU computing (requires nvidia card that supports compute capability 1.3): mathworks.com/discovery/matlab-gpu.html
  • Joel B
    Joel B almost 13 years
    Why is it best to parallelize only the innermost loop? I've read the opposite from all the literature I've looked at for .NET. Generally, parallelizing the big outer loop incurs the overhead of creating threads less frequently than parallelizing the inner loop would.
  • Jonas
    Jonas almost 13 years
    @Joel B: You want to parallelize a loop that does a lot of iterations, since running a loop that does, say, 500 iterations on a 1024-core cluster is quite a waste. Of course, the core of the parallelized loop should take more time than thread creation overhead. However, this is almost always true in Matlab, since you require a lot fewer loops, and thus the calculation within even the innermost loop is quite heavy.
  • Joel B
    Joel B almost 13 years
    But what about for just a quad-core single PC? Probably doing the outer loop makes more sense there? Would it be fair to say that parallelizing the inner loop is better if I have a huge computer cluster at my disposal, but the outer loop better to parallelize if I only have a single, multi-core PC?
  • Jonas
    Jonas almost 13 years
    @Joel B: Even then it's better to parallelize a loop with many iterations. If you run 4 parallel threads, one of them will be slower because its core is also running OS jobs. If you're running parallel jobs on cores with different speeds, you want to avoid having to wait for one of the cores to finish the last iteration. This is a lot easier if there are lots of iterations. Again, you have to balance the possible waiting time against the parallelization overhead. With Matlab, that overhead has never been a problem for me, .Net might be different.