For abersailbot, I've been working on automation of provisioning images to run on the main raspberry pi inside the boat. In the past, this has been a primitive bash script which installs packages and configurations files with a dubious amount of calls out to apt, cp and sed. Wanting to make this a little more sophisticated, I started writing a set of ansible playbooks.

For testing other repositories, I normally always use https://travis-ci.org. If you're not familiar, Travis CI offers free CI resources for open source repositories. The VMs it boots are all Ubuntu on x86_64, however. In order to test things on raspbian, some hacks would be needed.

Options I considered:

1. No testing (no tests? What madness!)
2. ARM chroot
3. Raspbian fat Docker container
4. Raspbian image booted under QEMU

After some playing around, the QEMU route seemed like the best option. If you'd like to do the same, the steps I took are fairly simple:

First off, we need to get a disk image. The images provided from the Raspberry Pi site need some small modifications to run under QEMU, so we need to mount and edit it. I'm using the 2015-11-21-raspbian-jessie-lite.img image from https://www.raspberrypi.org/downloads/raspbian/.

This image contains two partitions, a small boot partition (FAT) which contains a few config files and a kernel, and the main ext4 root partition. The latter is the partition we're interested in. You can inspect the image with fdisk to show these:

~/g/dewi (master) $ fdisk -l 2015-11-21-raspbian-jessie-lite.img
Disk 2015-11-21-raspbian-jessie-lite.img: 1.4 GiB, 1458569216 bytes, 2848768 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb3c5e39a

Device                               Boot  Start     End Sectors  Size Id Type
2015-11-21-raspbian-jessie-lite.img1        8192  131071  122880   60M  c W95 FAT32 (LBA)
2015-11-21-raspbian-jessie-lite.img2      131072 2848767 2717696  1.3G 83 Linux

Note the start of the partition we're interested in (131072). This is the offset from the start of the image in terms of number of sectors. The sector size is 512, so the actual offset in terms of bytes is 131072 × 512 = 67108864.

Armed with this knowledge, we can mount the image and edit files on it interactively.

First, create a directory to mount the image:

$ mkdir raspbian-jessie-mount-point

then mount the image:

$ sudo mount -o loop,offset=67108864 2015-11-21-raspbian-jessie-lite.img \
  raspbian-jessie-mount-point

Now it's time to comment out a bunch of things.

1. Open raspbian-jessie-mount-point/etc/ld.so.preload in an editor of your choice and comment out all lines.
2. Do the same and comment out all lines related to mmccblk in raspbian-jessie-mount-point/etc/fstab

After these edits have been made, unmount the image:

$ sudo umount raspbian-jessie-mount-point

The modified image can now be saved and backed up somewhere. I took a copy and compressed it to save on space:

$ xz 2015-11-21-raspbian-jessie-lite.img

Now we have an image to boot, we need a kernel to run. Sadly, a modified kernel is required, since QEMU does not support raspberry pi hardware. Luckily the work of patching and building has already been done by someone else. https://github.com/polaco1782/raspberry-qemu appears to work well.

Clone that repository and copy kernel-qemu:

$ git clone git@github.com:polaco1782/raspberry-qemu.git
$ cp raspberry-qemu/kernel-qemu .

At this point, booting should be possible. To test, first install qemu-system-arm.

If you're on Fedora:

$ sudo dnf install qemu-system-arm

Or Debian/Ubuntu:

$ sudo apt install qemu-system-arm

Then run:

$ qemu-system-arm \
    -kernel kernel-qemu \
    -cpu arm1176 \
    -m 256 \
    -M versatilepb \
    -no-reboot \
    -serial stdio \
    -display none \
    -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" \
    -net user,hostfwd=tcp::10022-:22 \
    -net nic -hda \
    2015-11-21-raspbian-jessie-lite.img

Note the option -net user,hostfwd=tcp::10022-:22. This forwards port 22 on raspbian to 10022 on the host, allowing us to ssh into the booted VM.

You should see some output similar to:

pulseaudio: pa_context_connect() failed
pulseaudio: Reason: Connection refused
pulseaudio: Failed to initialize PA contextaudio: Could not init `pa' audio driver
ALSA lib confmisc.c:768:(parse_card) cannot find card '0'
ALSA lib conf.c:4248:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4248:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1251:(snd_func_refer) error evaluating name
ALSA lib conf.c:4248:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4727:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM default
alsa: Could not initialize DAC
alsa: Failed to open `default':
alsa: Reason: No such file or directory
ALSA lib confmisc.c:768:(parse_card) cannot find card '0'
ALSA lib conf.c:4248:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4248:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1251:(snd_func_refer) error evaluating name
ALSA lib conf.c:4248:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4727:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM default
alsa: Could not initialize DAC
alsa: Failed to open `default':
alsa: Reason: No such file or directory
audio: Failed to create voice `lm4549.out'
Uncompressing Linux... done, booting the kernel.

Test if a login works:

$ ssh localhost -o StrictHostKeyChecking=no -p 10022 -l pi

The password should be the default raspberry, unless you changed it earlier.

If this all works, you can start writing your .travis.yml to boot this VM on Travis CI runs. You should first enable travis commit hooks on your repository (read the getting started guide), then start writing a configuration file. A simple version of the config file to start off:

language: generic
sudo: true
dist: trusty

before_install:
    - sudo apt-get update
    - sudo apt-get install qemu-system-arm wget xz-utils sshpass

install:
    - wget "https://super-secret-location.com/2015-11-21-raspbian-jessie-lite.img.xz"
    - unxz "2015-11-21-raspbian-jessie-lite.img.xz"
    - git clone https://github.com/polaco1782/raspberry-qemu.git

before_script:
    - qemu-system-arm -kernel raspberry-qemu/kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" -net user,hostfwd=tcp::10022-:22 -net nic -display none -hda 2015-11-21-raspbian-jessie-lite.img &
    - ./test/wait-for-ssh

script:
    - sshpass -p raspberry ssh localhost -o StrictHostKeyChecking=no -p 10022 -l pi "cat /etc/os-release"
    # whatever you want to test here

You should note this requires a mirror of the image created in the earlier steps. Each CI run will have to download and decompress the image. With Travis Pro, I believe you might be able to do something different and cache it, but I didn't investigate this.

This runs qemu-system-arm in the background. In order to wait until the VM is ready, I made a crude script named wait-for-ssh to ping the ssh port until it responded:

#!/usr/bin/env bash

while true; do
    if nc -z localhost 10022 ; then
        echo "Found something on port 10022!"
        exit
    else
        echo "Nothing found on port 10022, sleeping..."
        sleep 10
    fi
done

When this is done, you can actually test the stuff you wanted to test in the first place! In the example .travis.yml, I do a sanity check and cat /etc/os-release, expecting something like:

PRETTY_NAME="Raspbian GNU/Linux 8 (jessie)"
NAME="Raspbian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Now you have a little Raspbian VM booted on every commit. Run some ansible or puppet against it and you're done. Happy testing!

As part of Google Summer of Code, I've been working with the CentOS project to create a delivery and build system to provide easy and automatic delivery of patches to a running linux kernel. CentOS 7 currently has support for live patching a running kernel (via the kpatch kernel module and the surrounding userland tools), but crafting a patch and applying it is a currently a very manual process. The overall aim of the project is to make applying kernel fixes an easy and automatic process, which will make it easier to keep a CentOS installation more secure by without having to schedule downtime.

As the first part of this project, I've been writing kpatch-package-builder, a tool which generates RPM packages containing a kernel module which patchs the currently running kernel. These distributable packages contain a single kernel patch, so standard package management tools, such as dnf and yum can install and manage them.

This first version is pretty simple. You create a patch against the source of the kernel you want to modify, then run kpatch-package-builder with the patch name as the last argument. The output can either be a built .rpm file, or the raw spec.

By default kpatch-package-builder outputs an RPM .spec file, which can be used to build the RPM. This is a very simple spec, which just specifies the name of the package, description, and other common package metadata. The name of the package currently is based off the patch filename with a prefix of kpatch-module-, which allows these packages to be simply distinguished. Future versions will have a series of virtual dependencies to prevent patch collisions, but that's something for a future post.

When given the -b or --build-rpm option, kpatch-package-builder will invoke rpmbuild and build the package in place. The makes the kernel patch to distributable package a single step.

As an example, creating a package from a patch file:

[vagrant@localhost ~]$ kpatch-package-builder -b livepatch-test.patch
kpatch-package-builder: generating spec file...
kpatch-package-builder: building RPM...
kpatch-package-builder: writing spec file to /tmp/kpatch_3I6mHJ.spec...
kpatch-package-builder: writing kpatch-module-livepatch-test-1-1.x86_64.rpm...

The default metadata associated with that generated package:

[vagrant@localhost ~]$ rpm -qip kpatch-module-livepatch-test-1-1.x86_64.rpm
Name        : kpatch-module-livepatch-test
Version     : 1
Release     : 1
Architecture: x86_64
Install Date: (not installed)
Group       : System Environment/Kernel
Size        : 281661
License     : GPLv2
Signature   : (none)
Source RPM  : kpatch-module-livepatch-test-1-1.src.rpm
Build Date  : Wed 24 Jun 2015 15:26:46 EDT
Build Host  : localhost
Relocations : (not relocatable)
Summary     : kpatch livepatch module
Description :
Package generated from livepatch-test.patch by kpatch-package-builder

The contents of the package:

[vagrant@localhost ~]$ rpm -qlp kpatch-module-livepatch-test-1-1.x86_64.rpm
/var/lib/kpatch/3.10.0-229.el7.x86_64/kpatch-livepatch-test.ko

The current options available:

[vagrant@localhost ~]$ kpatch-package-builder -h
usage: kpatch-package-builder [-h] [-v] [-o FILE | -b] [-k VERSION] [-a ARCH]
                              [--set-release NUM] [--set-version NUM] [-d]
                              PATCH

Build an RPM package or an RPM spec file to install and manage a kpatch
livepatch module.

positional arguments:
  PATCH                 patch file from which to build the livepatch module

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -o FILE, --output FILE
                        name of output spec file
  -b, --build-rpm       build an RPM package
  -k VERSION, --kernel VERSION
                        target kernel version to build the livepatch module
                        against. Defaults to the currently running version
  -a ARCH, --arch ARCH  architecture to compile the patch against
  --set-release NUM     package release version
  --set-version NUM     package version number
  -d, --debug           print debug information

Usage examples:

Build an RPM package for a given patch:

    $ kpatch-package-builder --build-rpm module.patch

Generate a spec file to later build into an RPM:

    $ kpatch-package-builder --output module.spec module.patch

If you want to test this or use it yourself, the code lives at https://github.com/centos-livepatching/kpatch-package-builder.

Vagrant is now packaged in Fedora 21 and later versions, which is great news. No more messing around with the horror of rvm!

After installing vagrant and the libvirt plugin with yum:

$ sudo yum install vagrant vagrant-libvirt

I got some syntax errors from Vagrant:

$ vagrant up
/usr/share/vagrant/lib/vagrant/pre-rubygems.rb:19:in `require_relative': /usr/share/vagrant/lib/vagrant/bundler.rb:217: syntax error, unexpected tPOW (SyntaxError)
    def internal_install(plugins, update, **extra)
                                            ^
/usr/share/vagrant/lib/vagrant/bundler.rb:298: class definition in method body
/usr/share/vagrant/lib/vagrant/bundler.rb:315: class definition in method body
/usr/share/vagrant/lib/vagrant/bundler.rb:368: syntax error, unexpected keyword_end, expecting $end
    from /usr/share/vagrant/lib/vagrant/pre-rubygems.rb:19:in `<main>'

This was caused by rvm having an old version of ruby installed, which is what was installed when I installed vagrant on this machine initially:

$ ruby --version
ruby 1.9.3p545 (2014-02-24 revision 45159) [x86_64-linux]

Luckily you can just:

$ rvm implode

and everything will be happy, where happy is defined by vagrant running and having the fewest versions of ruby installed.

I've since done my GCSE and A-level exams, and am now half way through university.

Some sailing robots made with friends from my university:

Old posts begin here:

I've recently been playing with the docky python bindings, and started by writing a helper to interact with Vagalume, a lightweight last.fm client (I later added it to a project on launchpad). The vagalume DBus methods and signals are mostly undocumented, but they can be found lurking around after a quick look at some of the source code.

import dbus
import gobject
from dbus.mainloop.glib import DBusGMainLoop

class Vagalume(object):
    def __init__(self):
        dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
        self.bus = dbus.SessionBus()

        self.player = self.bus.get_object('com.igalia.vagalume',
                                          '/com/igalia/vagalume')

        self.bus.add_signal_receiver(self.song_changed,
                                     dbus_interface='com.igalia.vagalume',
                                     signal_name='notify')

    def song_changed(self, *args):
        state = args[0]
        if state == 'stopped':
            #do something when the player is stopped
            self.stopped()
        elif state == 'playing':
            artist = args [1]
            title = args [2]
            album = args [3]
            #do something with the data here...
            self.playing(artist, title, album)

    def stopped(self):
        '''Run when music is stopped'''
        pass

    def playing(self, artist, title, album):
        '''Run when new song is played'''
        pass

if __name__ == "__main__":
    app = Vagalume()

    # gtk mainloop can be used if you're using this as part of a gtk app
    mainloop = gobject.MainLoop(is_running=True)
    mainloop.run()

You'll probably notice the method song_changed. This is a method which is registered with DBus and run each time the 'notify' signal is emitted from vagalume. This checks whether the notification is due to vagalume starting a new song or stopping the current one and runs either self.stopped or self.playing. These two methods by default do nothing, so subclassing Vagalume to make them do something useful is good:

class MyFancyVagalume(Vagalume):
    def stopped(self):
        print 'just stopped the beat'

    def playing(self, artist, title, album):
        print 'now playing "%s" by %s' % (title, artist)

To interact with vagalume do something like:

vagalume = Vagalume()

# do anything you want with dbus
vagalume.player.Play()
vagalume.player.Skip()
vagalume.player.LoveTrack()
vagalume.player.BanTrack()
vagalume.player.Stop()

take a look at the source (http://gitorious.org/vagalume/vagalume/blobs/master/src/dbus.h) for all of the functions available

A couple of days ago I needed to take screenshots of my linux desktop's screen and manipulate these captured images using Python Image Library (PIL). I wrote a small python class to do the work of taking the screenshot and returning a PIL Image object for use in the rest of the system I was writing.

The code ended up being simpler than I expected, mostly because GDK, the library used as an intermediary between GTK and the low-level window manager and display server commands, does all the heavy lifting.

import Image, gtk

class Screenshotter(object):
    def __init__(self):
        self.img_width = gtk.gdk.screen_width()
        self.img_height = gtk.gdk.screen_height()

        self.screengrab = gtk.gdk.Pixbuf(
            gtk.gdk.COLORSPACE_RGB,
            False,
            8,
            self.img_width,
            self.img_height
        )

    def take(self):
        self.screengrab.get_from_drawable(
            gtk.gdk.get_default_root_window(),
            gtk.gdk.colormap_get_system(),
            0, 0, 0, 0,
            self.img_width,
            self.img_height
        )

        final_screengrab = Image.frombuffer(
            "RGB",
            (self.img_width, self.img_height),
            self.screengrab.get_pixels(),
            "raw",
            "RGB",
            self.screengrab.get_rowstride(),
            1
        )

        return final_screengrab

if __name__ == '__main__':
    screenshot = Screenshotter()
    image = screenshot.take()