A perfSONAR User's Group

Reducing Raspbian Install To Minimum Therbligs With raspbian-ua-netinst

A significant bottleneck in the workflow, when deploying multiple Raspberry Pis is the SD card preparation and then the individual machine config. The default workflow, for the young experimenter market that the Raspberry targets, is to write an SD card, boot the Raspberry Pi from it, connect a keyboard and monitor and answer questions. Fun once or twice, but not 50 or 100 times.

Worthy of note — Raspberry Pi (et al) are popular as teaching devices in classrooms, so there are various guides to getting multiples set up. The NOOBS system, which offers a choice of several OS options (both local and netinstall) at set-up time, comes with the capability to do unattended installs. NOOBS, however, is based on RISCOS, and a little reading suggested that the changes I wanted to make (like specifying a local repository), were going to require me to spend time understanding RISCOS, and I just don’t see that as a skill I’m going to use much going forward.

After seeing what-was-what with the NOOBS bulk install option, I was left wanting more.

I ended up with raspbian-ua-netinst –

https://github.com/debian-pi/raspbian-ua-netinst

Which allows you to do a 5-second write per SD card, and then let the RPi install itself on next boot.

By fashioning a couple of custom scripts to tweak the install, and by adding or subtracting from the package list, one can develop a process that allows the preparation of a custom install card in 20 seconds or less, and the install process is handled by each Raspberry, which needs network (DHCP) and power to install itself.

The workflow I developed for our Raspberry Pi installs produces a Puppet-registered Raspberry in less than an hour. Once each node is up and set up with Puppet, it gets the same configuration as the production nodes, with change management, etc. In order to generate individual node inventory (and etc.) tracking, I used a Perl script to copy the files to each card, and to increment an install serial number and hostname for each one.

The UA Netinstall process wasn’t perfect when I took my snapshot in October 2014. I had to add an additional apt-get update and a firmware update/reboot to get the linux modules to match the kernel. Work on UA Netinstall seems to be still happening as of this writing,  so YMMV.

Most of the customization is done through two files, installer-config.txt, and post-install.txt.

My example installer-config.txt:

hostname=swarm001a
rootpw=VeryN1ceP4ssw0rd
mirror=http://host.example.edu/raspbian/
cdebootstrap_cmdline="--flavour=build --include=kmod,fake-hwclock,
ifupdown,net-tools,isc-dhcp-client,ntp,openssh-server,vim-tiny,
iputils-ping,wget,rsyslog,dialog,locales,less,cron,man-db,iptables,
ethtool,iperf,nuttcp,tcpdump,perl,puppet"

which sets a root passwd, hostname and package list, to add to the base install. It’s too bad that you can’t yet add an encrypted password hash, like in a RedHat kickstart install file.  Note that I built a local copy of the Raspbian repo on a web server to enhance install performance and avoid hammering the main Raspbian repos with many simultaneous installs.

And post-install.txt:

echo "if [ ! -f /var/lib/puppet/ssl/certs/ca.pem ]" > /rootfs/usr/local/bin/puppetize.sh
echo "then" >> /rootfs/usr/local/bin/puppetize.sh
echo " /usr/bin/puppet agent --test --waitforcert 30" >> /rootfs/usr/local/bin/puppetize.sh
echo " /usr/bin/puppet agent --enable" >> /rootfs/usr/local/bin/puppetize.sh
echo "fi" >> /rootfs/usr/local/bin/puppetize.sh

chmod +x /rootfs/usr/local/bin/puppetize.sh

echo "/usr/local/bin/puppetize.sh " > /etc/rc.local

echo "[main]" > /rootfs/etc/puppet/puppet.conf
echo "logdir=/var/log/puppet" >> /rootfs/etc/puppet/puppet.conf
echo "vardir=/var/lib/puppet" >> /rootfs/etc/puppet/puppet.conf
echo "ssldir=/var/lib/puppet/ssl" >> /rootfs/etc/puppet/puppet.conf
echo "rundir=/var/run/puppet" >> /rootfs/etc/puppet/puppet.conf
echo "factpath=\$vardir/lib/facter" >> /rootfs/etc/puppet/puppet.conf
echo "templatedir=\$confdir/templates" >> /rootfs/etc/puppet/puppet.conf
echo "prerun_command=/etc/puppet/etckeeper-commit-pre" >> /rootfs/etc/puppet/puppet.conf
echo "postrun_command=/etc/puppet/etckeeper-commit-post" >> /rootfs/etc/puppet/puppet.conf
echo "" >> /rootfs/etc/puppet/puppet.conf
echo "" >> /rootfs/etc/puppet/puppet.conf
echo "[agent]" >> /rootfs/etc/puppet/puppet.conf
echo "server = puppet.example.edu" >> /rootfs/etc/puppet/puppet.conf
echo "report = true" >> /rootfs/etc/puppet/puppet.conf
echo "pluginsync = true" >> /rootfs/etc/puppet/puppet.conf

Which gets each node to hook up with our Puppet master and from there, managing the group is easy.

On the first update, the puppet agent will install owamp and bwctl, as well as configuring NTP, etc.

So, for a many-node deployment, UA Netinstall can be a a very handy tool. The next logical step is to be able to remotely scratch install, as one can with a PXE-supporting platform. Since UA Netinstall begins by writing files into a FAT32 partition, and then shrinks the FAT32 to become the /boot partition, it may be possible to write a new UA Netinstall to that partition and set a flag to self-re-install on reboot.

Comments are currently closed.