About News Download Docs Bugs Contact

Distributed compilation




Distributed compilation with distcc

Distributed compilation in Source Mage allows a cast to spread the load of a build over several computers. This can greatly increase the speed of casting spells.



On the computer which should use distributed compilation run sorcery command and go to the Parallel/Cached/Distributed Compiling menu under Option, enter the network hostnames or IP-addresses of the volunteers in the DISTCC_HOSTS text field. Don't forget to add localhost if you want it to help.

Try casting a spell now and hopefully the compilation will be spread among the volunteers you specified.

Warning: only enable distributed compilation on a trusted network – no access control is done.

Cross compilation with distcc

A cross compiler for the desired target architecture (we will assume it's for an x86-64 on an x86 machine and located under /opt/x86_64-unknown-linux-gnu.
Have "distcc" spell installed and working either using the init script or the xinetd to run distcc.

For more info on cross compilers see http://www.kegel.com/crosstool/.

Steps on the client

1. Set up a "masquerade directory":

# mkdir /usr/lib/distcc/bin
# cd /usr/lib/distcc/bin
# ln -s ../../../bin/distcc gcc
# ln -s ../../../bin/distcc cc
# ln -s ../../../bin/distcc g++
# ln -s ../../../bin/distcc c++

2. Add the "masquerade directory" infront of your PATH variable:

This doesn't hurt normal compiling, just don't dispel distcc without removing this from your path.

Steps on the server

1. Create a user for distcc to run as:

# useradd distcc

2. Make sure your cross tools bin directory has a gcc, cc, g++ and c++ symlinks to the appropriate binaries:

# cd /opt/x86_64-unknown-linux-gnu/bin
# ln -s x86_64-unknown-linux-gnu-gcc gcc
# ln -s x86_64-unknown-linux-gnu-gcc cc
# ln -s x86_64-unknown-linux-gnu-g++ c++
# ln -s x86_64-unknown-linux-gnu-g++ g++

There isn't any need to link the ld, ar, cpp or as distcc is a distributed compilation tool not a distributed linker, C preprocessor, assembler or archiver ;-)

3. Modify the /etc/sysconfig/distcc or /etc/xinetd.d/distccd scripts appropriately to make distccd run as the distcc user:

4. Give the user the appropriate path:

Make sure the /etc/passwd has the users shell as /bin/sh:

# su distcc
$ cd
$ cp /etc/skel/.bash_profile ./
$ vim .bash_profile

and add export DISTCCD_PATH="/opt/x86_64-unknown-linux-gnu/bin:$PATH" to the file after the PATH variable is set.

5. Restart your distcc server: