This Thursday we installed apt-cacher-ng on our project’s webserver. The idea was to test [under a ‘relative amount of stress’] both:
- how fabric performs parallel execution
- how apt-cacher-ng copes with getting the packages, caching them, and providing upon request, in parallel
As ‘relative amount of stress’ we set: parallel installation of Gnome onto 20 Ubuntu 11.04 machines. As a result of running our fabfile, which included installing gnome on all the 20 workstations in the LAN nothing happend, i.e. no gnome installed. From the output we saw that packages could not be found, apt-cacher-ng was crashing every time we tried to install gnome on all the machines. Changing its configuration to provide up to 25 parallel threads did not help.
The workaround which did help was to first run the installation of gnome on one machine, and only then (after the operation was completed) to let fabric install gnome on all machines in parallel.
Conclusion: apt-cacher-ng could not at the same time manage fetching packages, caching them and serving the 20 machines, however, it managed to fetch packages, cache them and serve one machine and to then serve [from its cache] another 19 machines in parallel.
This leaves us with either / or:
- constructing our fabfile so that every time we want to install packages in parallel we first do it on one machine and then on rest
- instead of apt-cacher-ng use some other tool for proxy caching e.g. our project supervisor Tero Karvinen was suggesting to use Squid, a well-known, well-developed, well-supported, and widely used web-cache / proxy server.
The group has decided on the latter, so tests with Squid have begun.