How to speed up compilation of Ubuntu apps (make, cmake, gcc)

Question

I am compiling some programs here and I have 4 cores. Is there a way to tell make, cmake or gcc to compile using all cores or something to that affect?

Solution

If a package supports it you can use the -j flag to allow parallel jobs running, e.g.:

make -j8

More details on this flag can be found in the Stackoverflow question Why does make -j perform better when it is passed a number larger than the number of cores available?.

Distributed compilation

If you have multiple machines, give distcc a go. On the involved machines, sudo apt-get install distcc. Assuming that your build machine is 192.168.1.1:

• on the helper machines, run:

  sudo distccd --log-file=/tmp/distccd.log --daemon -a 192.168.1.1
  

• On the build machine, before running configure or cmake you have to specify hosts that you want to use for the build process. Optionally, specify the number of simultaneous jobs after a slash (defaulting to 4):

  export DISTCC_HOSTS='localhost/4 192.168.1.2/8 192.168.1.3/8'
  

  Make the compiler use distcc:

  export PATH="/usr/lib/distcc:$PATH"
  

  Now configure or cmake the application and build with:

  make -j$(distcc -j)
  

  Note that if you've put /usr/lib/distcc twice in your PATH, it'll fail. Be sure to set /usr/lib/distcc only once in your PATH.

For more details, see the manual pages for distcc(1) and distccd(1).

Comments

[Luis Alvarado]

OMG that J is good. I went from 15 minutes to less than 1. thanks L.

[Lekensteyn]

It can get faster with distcc, compilation of PHP was done in 2 minutes, the kernel was done in 3 minutes (three i5 machines)

[Luis Alvarado]

Will try with distcc when i get to work. Was asking a similar question about this yesterday. Maybe you can help with it here: askubuntu.com/questions/106810/…

[ams]

Running distccd as root seems like a bad plan. I've not used it for a few reasons but when I did I can't have used root because I didn't have permission on those machines.

[Lekensteyn]

@ams You do not need to run it as root. If you run it as root and want to switch users, use distccd --user nobody. Otherwise, it'll run under the user who executed it.

[xaxxon]

this answer needs more pump