Planet HantsLUG

Today my main machine was down for about 8 hours. Oops.

That meant when I got home, after a long and dull train journey, I received a bunch of mails from various hosts each saying:

• Failed to fetch slaughter policies from rsync://www.steve.org.uk/slaughter

Slaughter is my sysadmin utility which pulls policies/recipies from a central location and applies them to the local host.

Slaughter has a bunch of different transports, which are the means by which policies and files are transferred from the remote "central host" to the local machine. Since git is supported I've now switched my policies to be fetched from the master github repository.

This means:

• All my servers need git installed. Which was already the case.
• I can run one less service on my main box.
• We now have a contest: Is my box more reliable than github?

In other news I've fettled with lumail a bit this week, but I'm basically doing nothing until I've pondered my way out of the hole I've dug myself into.

Like mutt lumail has the notion of "limiting" the display of things:

• Show all maildirs.
• Show all maildirs with new mail in them.
• Show all maildirs that match a pattern.
• Show all messages in the currently selected folder(s)
  • More than one folder may be selected :)
• Shall all unread messages in the currently selected folder(s).

Unfortunately the latter has caused an annoying, and anticipated, failure case. If you open a folder and cause it to only show unread messages all looks good. Until you read a message. At which point it is no longer allowed to be displayed, so it disappears. Since you were reading a message the next one is opened instead. WHich then becomes marked as read, and no longer should be displayed, because we've said "show me new/unread-only messages please".

The net result is if you show only unread messages and make the mistake of reading one .. you quickly cycle through reading all of them, and are left with an empty display. As each message in turn is opened, read, and marked as non-new.

There are solutions, one of which I documented on the issue. But this has a bad side-effect that message navigation is suddenly complicated in ways that are annoying.

For the moment I'm mulling the problem over and I will only make trivial cleanup changes until I've got my head back in the game and a good solution that won't cause me more pain.

It was a great pleasure to see the Reduced Shakespeare Company again last week. They are currently touring the UK with their show “The Complete Works of Shakespeare (abridged) (revised)” before they take up residence at the Leicester Square Theatre over the summer. This was the third time I’ve seen the Shakespeare show, albeit the first time in the “(revised)” form.

I’ve seen some of their other shows (I wrote about seeing the Complete World of Sports last summer) but have always had a soft spot for the Shakespeare show: It’s funny, but in an incredibly endearing way. The central concept is simple: Three people (Americans! Shock, horror) try to perform all of Shakespeare’s plays in an hour and a half, without realising how impossible their task is.

You don’t have to be familiar with Shakespeare to enjoy the show, although a little GCSE-level knowledge of a play or two helps. For the most part the updates in this revised version are subtle, work well and make sure that the show appeals to everyone.

I was lucky enough to get plucked from the audience by Matt Pearson (right) to run around on stage like an idiot. This means I can chalk up playing Ophelia’s ego alongside a pig and a urine tester in other RSC productions. As I got onto stage Matt Rippy (second from right) managed to work into the melee of dialogue that was flying around that he recognised me from Twitter!

I chatted to the guys briefly afterwards, and got to recommend a local curry house to Gary Fannin (left). One of the great things about RSC shows is that they differ depending on who is performing in them. This cast are great guys and work really well together, so get along to see them.

Lumail <http://lumail.org> received two patches today, one to build on Debian Unstable, and one to build on OpenBSD.

The documentation of the lua primitives is almost 100% complete, and the repository has now got a public list of issues which I'm slowly working on.

Even though I can't reply to messages I'm cheerfully running it on my mail box as a mail-viewer. Faster than mutt. Oddly enough. Or maybe I'm just biased.

First, it needs to be said. Automatic tracing and dumping data into Openstreetmap is not a good idea. This page is something I have been playing with as an aid to manual edits.

This is a simple summary of the steps I used. If you need more information, then you probably should not be doing this.

Source Data

Grab some data for the area of interest from Ordnance Survey.

The Gimp

Find the tile required and open in The Gimp. The following steps should isolate the buildings.

• Select by colour, threshold set to 26, pick the centre of a building, avoiding the antialiased edges.
• Sharpen selection.
• Fill whole selection with black
• Invert Selection
• Fill whole selection with white.
• Save image in bmp format.

Potrace

Make sure that you have the latest potrace that does geojson. With the required tif image from the OS data, run to following command:
potrace -b geojson -L XXXXXX -B YYYYYY -O 1 -a 0 tile.bmp
Replace XXXXXX and YYYYYY with the appropriate offsets for the tile being worked on. The required data is in the package downloaded from OS.

Quantum GIS

• Create a new project with CRS OSGB 1936 / British National Grid. EPSG:27700.
• Load a new vector layer, select the json file from potrace and make sure the CRS is as above. Make sure that the scale is making sense. It would be possible to load another layer of known good data as a check. Be sure to allow on the fly CRS transforms if the data you check against is not OSGB 1936.
• Save the layer as a shapefile. Use the same CRS as the layer for the shapefile.

JOSM

The shapefile can now be loaded into JOSM. The polygons should line up well with the OS OpenData StreetView background images. However, there will still be a lot of manual cleaning up required. Extra nodes need to be deleted and squaring up done.

I am not sure if this actually makes much improvement over simply clicking over the background imagery by hand. Maybe someone else can improve this process a bit more. This is really written as a reminder to myself incase I come back to this later.

All this is done with free software. The initial idea came from the openstreetmap wiki. There is also a python program that can do this called Mapseg, but it is not very fast on my little computer.

A massive marathon has resulted in my lumail mail client working well.

Functionally the application looks little different to the previous C-client, but it is a lot cleaner, neater, and nicer internally.

The configuration file luamail.lua gives a good flavour of the code, and the github repository has brief instructions.

Initially I decied that the navigation/index stuff was easy and the rest of the program would be hard; dealing with GPG-signatures, MIME-parts, etc.

But I'm stubborn enough to keep going.

If I can get as far as reading messages, with MIME handled properly, and replying then I can switch to using it immediately which will spur further development.

I'm really pleased with the keybinding code, and implementing the built-in REPL-like prompt was a real revelation. Worht it for that alone.

The domain name lumail.org was available. So I figured why not?

About a year ago I migrated all my workstations, laptops and netbooks to Arch Linux. Since then, I've setup Arch Linux on a Raspberry Pi and this server was also recently migrated to Arch Linux.

I've had no major issues issues during the last year and have upgraded through five major Linux kernels, transistioned to systemd and upgraded from Gnome 3.2 to 3.8.

Although I have been disciplined about merging .pacnew files frequently, during the upgrades and my many experimentations I have packages installed that I no longer require and obsolete files kicking about.

After the upgrade to Gnome 3.8 I decided to clean up a little. I rarely dip into the AUR, but when I do I always use packer to clearly seperate what is official from what is not.

Finding what is installed

The following commands are useful for identifying installed packages based on where they were installed from. The package lists generated from the commands below can be quite big but often highlight packages that I know I'm no longer using nor require.

Listing installed packages

List packages installed from the official repositories.

pacman -Qq | grep -Fv -f <(pacman -Qqm)

List packages installed from the AUR.

pacman -Qqm

Listing installed packages by size

Use pacsysclean to list installed packages sorted by size, it helps identify large packages that are no longer required which can the be manually uninstalled.

Listing orphaned packages

List ophaned packages install from the official repositories.

pacman -Qqtd | grep -Fv -f <(pacman -Qqtdm)

List ophaned packages from the AUR.

pacman -Qqmtd

Getting package information

Get package information for a package in the official repositories.

pacman -Si <package>

Get package information for a package in the AUR.

packer -Si <package>

Removing orphaned packages

Removing ophaned packages manaully can be very time consuming, but is by far the safer option. However, I decided to take a brave pill a uninstall all orphaned packages automatically.

Remove all ophaned packages installed from the official respositories.

sudo pacman -Rs `pacman -Qqtd | grep -Fv -f <(pacman -Qqtdm)`

Remove all ophanced packages install from the AUR.

sudo pacman -Rs $(pacman -Qqtdm)

Re-installing what you do need

When you do something scary like removing all the obsolete packages automatically, then you should really make sure you do have everything install that you require.

Re-install 64-bit base

sudo pacman -S --needed `pacman -Sqg base multilib-devel | grep -v gcc-libs | tr '\n' ' '`

Re-install 32-bit base

sudo pacman -S --needed `pacman -Sqg base base-devel | tr '\n' ' '`

Reinstall the groups required for a Gnome 3 desktop.

sudo pacman -S --needed `pacman -Sqg gnome gnome-extra telepathy | tr '\n' ' '`

Install all missing dependencies for packages in the official repositories.

sudo pacman -S --needed `pacman -Si $@ 2>/dev/null | awk -F ": " -v filter="^Depends" \ '$0 ~ filter {gsub(/[>=<][^ ]*/,"",$2) ; gsub(/ +/,"\n",$2) ; print $2}' | grep -v smtp- | sort -u`

Install all missing dependencies for packages in the AUR. This will re-install even if the package is already installed. I can't be arsed to filter it out for a one liner.

sudo packer -S --noedit --noconfirm `packer -Si $(pacman -Qqm) 2>/dev/null | awk -F ": " -v filter="^Depends" \ '$0 ~ filter {gsub(/[>=<][^ ]*/,"",$2) ; gsub(/ +/,"\n",$2) ; print $2}' | grep -v java- | sort -u`

Find files not associated with a package

When packages are removed they may leave some files behind. The following will find all files not associated with a package. These files can not be automatically deleted, each entry requires assessment.

pacman -Qlq | sort -u > /tmp/db
sudo find /bin /etc /sbin /usr ! -name lost+found \( -type d -printf '%p/\n' -o -print \) | sort > /tmp/fs
comm -23 /tmp/fs /tmp/db

As with all sping cleaning chores, I got bored by this stage as my workstation was looking pretty tidy. Much of what is presented in this blog post is a rehash of what others have already contributed to the Arch Linux Wiki. I've just organised what "Works For Me ™" so I know what to do next year.

References

• https://wiki.archlinux.org/index.php/Pacman_Tips

The story chosen to represent the Peter Davison era of Doctor Who at the BFI’s 50th anniversary screenings was Caves of Androzani. It’s an odd choice, as it is unlike other stories being produced at the time. It’s a excellent story, an excellent script and very well directed. But it’s not fun. The few quips and jokes in the script raised only muted laughter from the audience and, apart from the two leads, every character is flawed. There are no heroes in the story and almost every character dies, an ignominious end awaiting each of them. There are no blazes of glory here. The Doctor doesn’t influence events particularly, he doesn’t right any wrongs. He just wants to cure Peri and get out of there alive. Even then, he only half manages it.

The story was introduced by Mark Gatiss. Half way through. composer Roger Limb talked about his time at the Radiophonic Workshop. By a happy coincidence, the music from Caves of Androzani has just been released on CD.

The discussion panel afterwards was lively. To say the least. After his brief appearance at last month’s screening, I hadn’t expected to see Matthew Waterhouse this month, but there he was. The BFI even showed Adric’s death scene while the panel was on the stage. There was a fair bit of good natured ribbing between Peter and Janet, and some less good natured ribbing of Matthew. But things settled down and Matthew raised some interesting points. A refreshing number of questions from the audience were directed at Graeme Harper, and a fair few people who thanked Peter for being “their Doctor”.

Afterwards a small group of podcasters gathered in the by-now-traditional car park stairwell to review the story and the panel. It will be available from The Doctor Who Podcast Facebook page very soon. You can still download our review of “Robots of Death”, the 4th Doctor BFI screening.