Linux development on Windows with Vagrant and VirtualBox

February 12th, 2014

I’m doing some Linux-based server development at the moment, but my work machine runs Windows 7. I reckoned the best way to do this was to run a Linux VM, and a friend put me on to Vagrant, which is a crossplatform tool for managing VMs. Put simply, Vagrant allows you to build a VM according to a script, so that your whole build environment can be created repeatably on demand. This is really useful when a colleague joins your project, since you don’t have to spend a whole day remembering all the things you installed to make your build environment, and debugging issues with it not working on their machine.

Vagrant comes from the world of *nix, so it’s command-line driven and (as ever) such software doesn’t always play nicely with Linux. Here’s what I did to get it working on Windows 7.

  • Don’t bother getting any guest OSs for VirtualBox at this stage.

Vagrant needs an SSH client to work properly, and Windows doesn’t include one by default. It suggests using Cygwin, MinGW or git to provide one. I chose to use MinGW, which is a minimal set of tools to make Windows behave a lot more like a GNU system.

MinGW has a package-based installer – you download the installer, tell it which packages you want, and then it downloads and installs them for you.

So:

  • get the latest MinGW installer from https://sourceforge.net/projects/mingw/files/latest/download and then hit “Install” when it runs.
  • Keep the default installation directory (C:\MinGW) because like nearly all *nix tools, MinGW hates filepaths with spaces in. Make sure “install graphical user interface” remains ticked.
  • The installer should then open the package manager GUI (“MinGW Installation Manager”.
  • Go to Installation->Update Catalogue to ensure you have the latest package list.
  • Close the update dialogue box when it has finished.
  • Under “basic setup”, click on the checkbox next to msys-base and choose “Mark for installation”. Do the same for mingw32-base and mingw-gcc-g++.
  • Now go to “All packages” and scroll down to “msys-openssh”. Find the “bin” class and again mark for installation.
  • Finally, go to Installation->Apply Changes, and the system will download and install everything you need.

Finally finally, you need to alter that PATH environment variable so that Vagrant can find the ssh client:

  • Go to Control Panel->System And Security->System->Advanced System Settings and on the Advanced tab click “Environment Variables”.
  • Under “System Variables” scroll down to “Path” and click “Edit”.
  • The Variable Value box will be too small to contain the values, so carefully press your End key and add ;C:\MinGW\MSYS\1.0\bin to the end of the path. Note the semicolon that separates each individual path.
  • Having done that, close all the dialogue boxes and get a command prompt window. Follow the Vagrant introductory tutorial, it should all work now!

An ideal environment for research

October 5th, 2013

I’ve been very fortunate to work in a number of research institutions over the years. These are places set up specifically for people to work on new discoveries and developments – in my case, mostly in engineering, though it applies equally to many other fields. I really, really love good research institutions, and this evening I’ve been musing on what would make an ideal one. Here are my thoughts.

Firstly, you want the right people. It is, after all, mostly about the people. You want people that are interested and committed, and who have a varied range of skills and experiences. You’ll want several flavours of researcher from different fields, and then to support them with enthusiastic and helpful support staff. However, don’t employ too many – I think that about 200 people is the largest practical size. Why?

Firstly, geography – you can fit 200 people and all their stuff into one sprawling building, or a cluster of interlinked ones. You really don’t want to be having more than one building, or worse a split site with half the institution several miles down the road. This results in two institutions and not one. Most people can get to know 200 people and remember their names (mostly) within a year or two, given the right environment. Any bigger than this and you start to think of your colleagues as functions (“Purchasing”, “Geophysics”, “IT Support”) rather than as people, which really hinders collaboration.

If possible, you should provide opportunities for people to have casual conversations. These provide the vital cross-connections between people working in different offices. The key to ensuring casual conversations are two concepts from the world of retail – footfall and dwell time. Footfall is the number of people who pass through a space. If all your offices radiate from a central lobby, that lobby will always have high footfall as people move about the building, making it more likely that two people will meet in the lobby. However, if your offices are arranged into corridors by department, with the corridors interconnected at the central lobby, most people will only move about within their departmental corridor and the central lobby will be mostly quiet. In this situation, you can improve the chances of people meeting by increasing the dwell time in the central lobby. Put a coffee machine, a tea kettle. or a water cooler, and some chairs and tables in the lobby. People passing through will be encouraged to stop by, and people wanting coffee or water will come to the lobby deliberately and hang around. Increased dwell times mean more casual conversations.

Another great place to have casual conversations is in the canteen. You should definitely have a canteen – you don’t want your staff to disperse at lunchtime and not talk to one another, or worse to sit in their offices eating packed lunches whilst surfing the web. You should offer a choice of various types of food. You should allow people to eat their packed lunches in the canteen if they’d rather bring their own. You should have only just enough chairs and tables, so that people have to sit with other people that they don’t know when the place is busy. I think that having rectangular tables that seat four, that can easily be moved around, should be encouraged – these tend to then form a few long refectory tables for the gregarious types, whilst also providing quieter tables elsewhere. Make sure that the food is good and not too expensive. Make sure your catering staff are enthused and motivated – and perhaps even consider employing them directly rather than contracting out. For bonus canteen action, serve breakfast snacks mid morning and cake in the afternoon to encourage people to go, eat, talk and discuss their work with one another. If possible, make your canteen a through space, with multiple entrances to different parts of the building, so that it forms a natural meeting-place.

Now, the introverts reading this will say “but I need a place to think!” which is indeed very true. People need space to think and concentrate at times, without distractions. So, I’d recommend that you provide a number of tiny “carrels” – little quiet rooms with a desk and a chair and plenty of light – that people can book to use when they need a small, private space to work. Providing access to carrels means that you can continue to put your staff in shared offices – perhaps 3-4 to an office – which makes for better group-working than individual offices.

You’re also going to want meeting rooms of various sizes. Again, make these bookable by everyone, rather than reserving them for particular departments.

Now, on to the thorny topic of support staff. There’s always a reticence to employ support staff because they’re a “fixed cost” and seen as “not contributing directly” to research output. This is a fallacy. You have brilliant researchers, you want to enable them to do stuff they’re good at and give the stuff they are lousy at to other people that are better at it. Speaking as an engineer, I would strongly recommend that scientists are discouraged from building their own equipment without help from engineers! Huge amounts of time and resource can be wasted while people reinvent wheels or build Heath Robinson apparatus because they’re unaware of techniques or equipment that come from other fields. Likewise, good technicians are worth their weight in gold. Good mechanical technicians can make things out of bits of metal in an afternoon that would take me a week. Good electronic technicians can wire up cabinets neatly, and solder delicate components without damaging them. Employ good technicians, pay them properly, make them feel valued.

Give your research staff a briefing on how to use the support departments effectively. This isn’t rocket science, but a surprising number of people don’t get it.  The simple rules for dealing with support departments are:

  • Be polite.
  • Ask nicely.
  • Explain clearly what you want, and when you want it by.
  • Be reasonable.
  • Say thankyou afterwards.

If you do this as a researcher, you will find that your support staff will go the extra mile to help you when you find yourself in a difficult situation. If you take them for granted, they’ll get jobsworthy.

As a manager, please don’t ask your support departments to charge internally for their time. This causes two levels of evil – firstly, the researchers go “how much!?” and then try to do the support task themselves badly, or circumvent it by some other method to avoid paying. Secondly, your support departments become less helpful, because the response to “can you help me with this?” becomes “what’s your charge code? it’ll cost you!”. By all means have your support staff keep job logs, so you can see which individuals/departments are making heavy demands on support departments if you think they are being abused.

Information is the lifeblood of research. You need to know what is being done elsewhere, and also crucially what has been done in your institution in the past. So, you need access to books, journals and conference proceedings. A lot can be obtained electronically but you probably will still need a library to keep physical media in. This also provides another useful quiet workspace. Please have everything in some sort of electronic catalogue or portal page so that staff can easily find out whether the institution has access to the particular paper they’re looking for. Consider partnering with a university library to buy information services from them, or to allow your staff to have “visiting scholar” access if required.

Almost as critically, make sure that you retain information within your institution. Knowledge that doesn’t get published in peer-reviewed journals should nevertheless be held onto within an internal publishing system. I’m strongly in favour of the BBC’s system for doing this, which is called the Technical Note. TNs can pretty much contain any content you like and be of any length, though most are 10-30 A4 pages. When a member of staff writes a TN, it is signed off by their Head of Group and circulated to all the other senior managers and anyone else involved in the work. It is also archived and catalogued. The distribution of TNs ensures that information flows between departments, via the managers, and crucially provides some helpful validation for the researcher in question when senior people meet them in the corridor and say “Oh, I found your TN very interesting!”. TNs are also a convenient, measurable deliverable thing that results from any piece of work – a few hours of exploratory research, a new proposal, the outcome of a brainstorm – all these things can be captured, archived, and distributed around the institution.

Provide great facilities, the best that you can afford. Make sure someone take responsibility for looking after them – research labs often suffer “crisis of commons” effects where valuable equipment gets damaged through lack of experience and training. Provide “general” lab space, and make sure that you avoid departmental turf wars over lab and bench space. If you are constrained for lab space (and who isn’t?), I suggest a leasing system – a bench is “leased” to one project for a given period (weeks or months) and the lease is then reviewed when it “expires”. If the project is continuing, a new lease can be issued. This avoids the problem of dead projects squatting facilities that are needed for new ones. Some sort of “warm storage” facility would be nice – then you can put equipment into store that may not be used for a few months, rather than have it sit there gathering dust in the lab.

Administration is necessary and helpful, and admin people should not be looked down upon as lesser beings. On the other hand, you should not allow your administrators to dictate business processes for their own benefit! Make your purchasing and budgeting systems streamlined. If possible, provide regular updates to budget holders about how much money they have committed from their budget to purchase requests and how much has actually been paid out against invoices. Most accounting systems I’ve seen only keep track of payments and not budgets, so nearly every manager I’ve ever worked for has run some sort of parallel accounting system in Excel to work out how much budget they have left. This is a waste of everyone’s time – build the budgeting into the purchasing system and make it work for everyone. Train your research staff in how to interact with the admin and business processes, and make sure you have guides to common procedures in a staff handbook or on an intranet.

To summarise – hire awesome people, help them work together, with excellent facilities and well-thought-out processes. Produce excellent research. Profit!


Music in the air

March 27th, 2012

“You know”, my dearly beloved said to me, “It must be possible to play music in the living room more easily!”. He knows I’m an obliging sort, and an engineer, and I fell for that one hook, like and sinker. He outlined the brief in the manner of the client who knows what he wants but doesn’t know how it can be achieved – “I don’t want to bend over my old iPod to choose tracks. I’d like to play music across the network from one of the computers. Oh, and I’d like to control the playlist from my phone.”

I promised that something could be done. But it would require time, and would almost certainly require money.

So, I surveyed the existing things I had to work with:

– a Fatman iTube iPod dock / amplifier and speakers
– a classic iPod from days gone by
– a MacBook with an iTunes library full of music
– a wireless network

It ought to be possible to make that work out – all I needed was a box of tricks that would connect to the Fatman’s line input and pull music over the network. I punched “network audio player” into Google and found myself adrift in a world of solutions that didn’t quite work…

Here are some that I considered:

Logitech Squeezebox. I know that Lorna is a big fan of Squeezebox and has them all over her house. But the entry-level Squeezebox Radio is £150 and you have to be running their software on a machine that’s on all the time.

– Apple’s AirPort Express. A funny old gadget this, basically a wireless access point that also can output a line-level audio stream and be a print server. They cost £79. The audio output works using Apple’s (inevitably proprietary) AirPlay system, so that locks me (as an Android user) out from the system. Dearly beloved boyfriend has an iPhone, so he’d be all right with it. Still, possible but not ideal.

Revo Mondo Wi-Fi. This is a little adaptor device that’s designed to plug into a hi-fi. It’s mostly intended as an internet radio, and has a basic two-line display. It’s about £90 from online retailers. The underlying technology comes from a firm called Receiva, and claims to support BBC on-demand radio content (i.e Listen Again / iPlayer radio) as well as live streams.

In the process of looking at all these bits of kit I kept coming across mentions of DLNA and UPnP. I did a bit more research and discovered that these standards offered me what I wanted – a flexible, cross-platform system. So, what are they? DLNA stands for the Digital Living Network Alliance, and it’s a standards body that provides standards for networked AV systems. UPnP is one of the underlying technical standards – it stands for “Universal Plug ‘n’ Play”. For the purposes of networked AV, there are three basic items in a DLNA system:

– a media server: this is a box full of media (music, video, photos) connected to the network
– a “control point”: this is a control device, that allows a user to choose media to play
– a “media renderer”: this is a device that can receive commands from a control point and content from a media server and play it.

Confusingly, a lot of products support different bits of the standard and don’t always tell you which ones! Anyway, here’s my approach:

– Media server: MacBook running Twonky 7.0. There is quite a lot of server software out there – they all have different quirks. Twonky is easy to get going with, but it’s somewhat irritatingly bound up with Twonky’s attempt to be a social media site. I’m planning to eventually switch to a DLNA-compliant NAS device with all our media on it.
– Control point: my Android phone (HTC Desire S) running BubbleUPnP. There is a free Twonky app for Android (and iOS), but I found it a bit clunky and it also doesn’t seem to be able to adjust the volume control on a remote renderer. Note that BubbleUPnP (and Twonky Mobile) can act as a server, control point and a renderer all on the same device.
– Renderer: I bought an Archos 35 Home Connect internet radio, which cost me £88 from Amazon. The Archos box is a strange concept – basically a tiny, cheap Android tablet given a pair of kitchen-radio speakers – but it seemed like just the thing for the job. It runs BubbleUPnP also, acting as a media renderer. Because it has a little touchscreen it’s possible to interact with it directly, but I’ve mostly been using my phone to control it remotely. Note that you need the full version – costs £3 – to make Bubble UPnP into a remote media renderer: once you’ve installed the licence app, go to Settings->Local Renderer->Allow remote control and then it’ll show up as a renderer on the network. After a false start yesterday (turns out Google Play gets confused if you buy the same app twice in quick succession for two different devices on the same account) the system is working nicely now. The Archos has a 3.5mm audio out, which I’ve hooked up to the Fatman amp in the living room.

Since the Archos is an Android device, you can make it run any app you like. You’ll need to install ArcTools from AppsLib to get the full Android Market / Google Play functionality though. I like to listen to BBC radio programmes via iPlayer, which works on the Archos. My only gripe is that iPlayer’s layout doesn’t work very well with the Archos’s landscape-only screen, and so you end up scrolling a lot to find the programme you want. Since the Archos has its own internal battery and speakers, I can bring it into the kitchen (disconnecting it from the amp) and carry on listening whilst I cook.
Its other feature is a little VGA webcam above the screen, so it could be used for Skype Video or even as a security camera!

The Archos unit is a “value-engineered” product, so don’t expect Apple-like levels of user experience. The resistive touchscreen is okay – I’ve used worse (TomTom, I’m looking at you) but not as good as you might be used to. That said, I think the Archos does the job I wanted pretty well. Let’s see how the “highly valued client” gets on with it!