This HOW-TO explains a quick and easy way to speed up NetBSD pkgsrc compiling on retrocomputers. It was written because there are altogether too many distcc how-tos which are complex, require special build environments or make assumptions without explaining what's going on and why. This is my attempt to simplify the process. So let's say you're a retrocomputing aficionado who enjoys running modern Unix on classic hardware. You're running NetBSD (after all, it's one of very few OSes which doesn't treat alternate architectures with disdain) but you cringe whenever Perl is updated because it'll take something like a week to update everything on your m68030. Sure, you already know how to crosscompile the OS, but that doesn't work for pkgsrc packages. On the other hand, you're not one to knowingly run insecure software, so waiting for binary packages for your platform isn't always an acceptable option. This guide will help you set up distcc to distribute pkgsrc compile tasks to another machine over your network. These days you can pick up a Raspberry Pi for $40 USD, a BeagleBone Black for $50 or a PogoPlug for $20 or less. I like PogoPlugs because they're physically robust and are really hard to accidentally unplug, whereas if you sneeze the wrong way around a Pi you might be waiting for it to finish a reboot (Ok, it's not really that bad, but you get the point). The BeagleBone Black and Raspberry Pi both have the obvious advantage of much greater memory than newer PogoPlugs (512 megs versus 128 megs), which may be important since creating the crosscompile toolchain for certain architectures can require more than 256 megs. Regardless of which device you get, or if you decide to use another system (amd64, PowerPC, sparc64, whatever), this guide will work all the same, just as this can be used to help a fast system with limited memory or one which compiles constantly. There are many guides out there meant to help people set up even faster machines to assist Raspberry Pi systems, but compared with an m68040 NeXT or a VAXstation 3100, a Raspberry Pi is incredibly fast. Considering the size, the amount of power and the cost, you might even dedicate a Pi or PogoPlug to each of your older computers or one device to several older retrocomputers. There are various ways to install NetBSD on the Pi, BeagleBone, PogoPlug, or others. Let's assume that you've already picked your favorite way, you've installed NetBSD and have a recent pkgsrc tree installed already (or have NFS mounted it if your storage is small/slow). You'll also need to get a NetBSD source tree of the same or similar branch as you're running on your retrocomputer (which can also be NFS shared since you'll only need it for a little while). System sources are in /usr/src and pkgsrc is in /usr/pkgsrc. In my examples I'm using a PogoPlug Mobile with a rather modest 16 gig USB flash drive. It has an 800 MHz StrongARM (Kirkwood ARMv5), 128 megs of memory and gigabit ethernet and cost $14 USD when Radio Shack still had them. See my article on how to install NetBSD without needing a serial console if you're interested (link will be provided). The retrocomputer this machine will assist is a VAXstation 4000/60 with 56 megs of memory. I'll refer to them as retro and modern. retro in these examples is at 10.12.26.190 and modern is at 10.12.26.189. Start by installing devel/distcc from pkgsrc on both retro and modern machines. If you'd prefer to use distcc 3.2rc1 instead of 3.1, remove patches/patch-ad, patches/patch-src_emaillog.c and patches/patch-src_dotd.c, then change PKGREVISION in Makefile to 0 and DISTCC_VERSION to 3.2rc1 in Makefile.common, then run make with NO_CHECKSUM=yes. While that's running, we'll compile a toolchain targetting retro on modern using NetBSD's build.sh. The locations below are chosen simply because they're easy to remember. Here's how we make a VAX toolchain. In /usr/src/: ./build.sh -O /usr/obj-vax -T /usr/tools -m vax tools Note that in this case, vax is the same for both machine and architecture. If running m68k, you'd select a machine (-m) name such as sun3, netx68k, mac68k or amiga, not m68k, but the path would end up being /usr/tools/m68k--netbsdelf/bin/. A little while later you'll have everything you need in /usr/tools/. If you look in /usr/tools/bin/, you'll see vax--netbsdelf-c++, vax--netbsdelf-gcc and friends, which are all StrongARM binaries which target VAX. Building the VAX toolchain on the PogoPlug took about two and a quarter hours, and on an eight core AMD about five minutes. Many online guides talk about symlinking stuff to the distcc binary or creating a wrapper script, but those methods are confusing and don't seem best for running a different architecture's toolchain while still calling the compilers with their standard names. Instead, I created a /usr/tools/vax/bin/ directory with four shell scripts named cc, gcc, c++ and g++. Sure, we could use one script for all four, but distcc only ever uses these four programs remotely so let's just make /usr/tools/vax/bin/ and make four scripts. Note that there is no vax--netbsdelf-cc. cc: #!/bin/sh exec /usr/tools/bin/vax--netbsdelf-gcc "$@" gcc: #!/bin/sh exec /usr/tools/bin/vax--netbsdelf-gcc "$@" c++: #!/bin/sh exec /usr/tools/bin/vax--netbsdelf-c++ "$@" g++: #!/bin/sh exec /usr/tools/bin/vax--netbsdelf-g++ "$@" Next, we will make a little shell script which we'll use to launch distcc on modern with the proper path. It looks like this: #!/bin/csh setenv PATH "/usr/tools/vax/bin:$PATH" distccd --allow 10.12.26.0/24 --listen 10.12.26.189 --port 3632 \ --daemon --jobs 2 --nice 0 \ --stats --stats-port 3633 \ --log-file=/var/log/distccd/vax.log The options in order are: --allow 10.12.26.0/24: Accept connections from this subnet. --listen 10.12.26.189: Listen on this specific address. --port 3632: This is the default port. Increment by two for each architecture if you want to use one device for multiple architectures. --daemon: Should be obvious. --jobs 2: Sets the number of simultaneous jobs distcc will accept. Documentations suggest number of CPUs plus two, but since we're not running more than one job at a time I selected two so that there will always be a distcc instance waiting even when closing up a previous instance. --nice 0: Sets the niceness of distcc tasks. We don't want them to be nice. Default niceness is 5. --stats: Allow querying of distcc statistics. Visit http://10.12.26.189:3633/ to see stats. Note that in order to see stats, the querying IP must be included in the allow statement above. --stats-port 3633: This is the default port for the http daemon. --log-file=/var/log/distccd/vax.log: Should be obvious. The distcc package requires the addition of the distcc user and group, so the log must be writable by one or the other (mkdir /var/log/distccd ; chgrp distcc /var/log/distccd ; chmod g+w /var/log/distccd). Go ahead and run the script to start the daemons. You should see three distccd processes. Since you started the compile of devel/distcc on retro before compiling the toolchain on modern, it should be done by now. You did start that first, right? Using distcc on retro is as simple as adding these lines to /etc/mk.conf: PKGSRC_COMPILER=distcc gcc MAKE_JOBS=1 DISTCC_HOSTS=10.12.26.189:3632 If your machine isn't quite as slow as a VAXstation or an m68030, you may want to use the following: PKGSRC_COMPILER=distcc gcc MAKE_JOBS=2 DISTCC_HOSTS=10.12.26.189:3632 localhost This will, for packages which can run parallel jobs, split the work between retro and modern machines. The reason I'm not recommending this as the default is that really slow machines will spend all their time just preparing for compiles and will be slowed by compiling anything at all by themselves. Now that the toolchain is built, distccd is running on modern and /etc/mk.conf has been edited on retro, we're ready to compile! To test the speed improvement, I did a small test with shells/tcsh. First I ran a "make fetch" ahead of time for the shells/tcsh package on a VAXstation 4000/60 with only 56 megs of memory. Then I ran "time make", both with and without distcc. Native compiling with no assistance: 4225.653u 958.195s 1:29:51.07 96.1% 0+0k 63+2055io 143pf+0w With distcc: 2196.750u 964.451s 59:58.47 87.8% 0+0k 0+2376io 48pf+0w This shows an almost halving of the time! Just to see what kind of a difference a faster compile system or running multiple jobs simultaneously would have, I tried with -j 2 and -j 4, and the differences in speed were less than 2%. Compiling on a 4 GHz AMD system versus an 800 MHz PogoPlug showed no noticeable differences at all, which suggests that the bottlenecks are all in the VAX. Enjoy!