It irked me a little that I could develop on one stack (OS X, Rubinius, Sqlite3), run continuous integration (CI) on another stack (Ubuntu, Ruby 1.8.7, Postgresql), and deploy into another stack (Gentoo, Ruby 1.9.2, MySQL). I think what irks and worries me is that there are three sets of differences to be aware of. A bug in production? Was it a missing test scenario or one of the many differences between production and CI environments?
So I think I have two solutions.
First, use a VM that matches the production environment. Each different production environment would mean another VM. If you are managing your own production environment, then all you need is the tools (described in this article) to recreate your production environment in a VM.
Second, use a clone of your production environment. That is, if you deploy to Engine Yard AppCloud then run CI in Engine Yard AppCloud; if you deploy to a single Ubuntu instance on Slicehost, the have another matching Ubuntu instance on Slicehost.
I’ll write about the first solution – using VMs – here, and I’ll write on the Engine Yard Blog about the solution for Engine Yard AppCloud customers. For AppCloud users life will be even easier because there are zero setup steps to ensure you have consistent environments. It’s been one of my favorite projects in the two months since I arrived at Engine Yard.
I’ll also introduce a CLI for talking to Hudson – one that assumes you are working on Ruby/Rails projects – and makes it really easy to get up and running with a server, your Rails/Ruby projects, and any VMs you need (optional for Hudson, but they are the point of my line of thinking).
And I’ll introduce Vagrant, a CLI for creating/managing/destroying VMs.
Oh, and I’ll introduce Hudson CI.
Ok, let’s fix all our CI problems in one go…
I needed a CI tool to allow tests to run inside VMs or on remote servers/VMs. Back in May I found and fell in love with Hudson CI. Fortunately it had support for “slaves”, and a way for CI jobs (your applications or rubygems) to select which slaves can be used. Hudson is also great because it is easy to try out (one click to install and launch), easy to configure, and has 350+ plugins.
CLI for Hudson
Charles Lowell’s hudson.rb project is a CLI for launching Hudson (it’s bundled inside the gem), and a set of CLI tasks to add/remove projects (jobs), add slaves, trigger builds and more.
gem install hudson
You can launch Hudson via:
hudson server # Default: --port 3001 open http://localhost:3001
It currently assumes you are working on Ruby projects, using bundler, and attempts to create a useful set of default steps for your CI jobs.
cd /some/rails3/app hudson create . --host localhost --port 3001 --template rails3
Hudson CI will automatically start running steps to install your application’s gems, database, and run your tests. When it does it on your local machine it’s not that impressive. When it happens on a fresh VM or slave node and you didn’t have to do anything to set it up, it’s awesome.
V is for Vagrant, VirtualBox and all things VM
If you can script it, then you can automate it. Fortunately, as a virtual machine VirtualBox is both scriptable and FREE! Secondly, if you have a VM, you’ll need to script its setup/provisioning; so fortunately there exists chef and puppet, amongst others.
Thirdly, if you are really really really lazy, like me, you will want Vagrant. Created by Mitchell Hashimoto and John Bender, Vagrant is a tool for building and distributing virtualized development environments. Everyone in your team have their own OS/configuration? Got Windows users on your team? Use Vagrant.
Once you have VirtualBox installed, getting any of your projects live within a VM is trival:
gem install vagrant vagrant box add base http://files.vagrantup.com/lucid32.box vagrant init vagrant up vagrant ssh $ cd /vagrant/ $ ls -al
You will now see the contents of your project folder, but from within the VM instance! They are linked – a change to either is automatically reflected inside and outside of the VM.
In this example, the Guest OS being downloaded and installed into VirtualBox (note: without any GUI) is Ubuntu Lucid, but you could use any VirtualBox packaged unix system.
For wonderful guided tour of Vagrant see the Getting Started video.
Hudson CI + Vagrant = Perfect CI
Whether you choose to use VMs for a production-like development environment (a good idea for anyone, a wonderful idea for Windows developers who are very far removed from their production experience), it is a very good idea to have a CI environment as similar to your production environment as possible. Here, we want a VM instance with all the components/utilities/rubies set up that you have in production.
I have created an example Rails application that is setup to use Vagrant for a CI slave VM.
Once you have the VM instantiated, you add it to your Hudson CI master (either the localhost one above or your remotely hosted server) with the CLI:
hudson add_node localhost --name "VM" --label railsapp-vagrant ...
(See the example Rails application for the other flags I used to get this working). It is now available to all Hudson jobs; but has a label “railsapp-vagrant” to allow jobs (Hudson’s name for a project) to specify that slave node specifically.
To add the Rails application to Hudson CI, and force it to run the tests in this VM:
hudson create . --template rails3 --assigned-node railsapp-vagrant
If you visit the Hudson master (at
http://localhost:3010 in the example) you will see the job automatically running (“building”) within your VM. It will use bundler to install the gems, and run all the tests.
I know, CI is historically a pain in the arse. When CruiseControl was the only CI kid-on-the-block, it was standard for people to respond “4 days” to the question “How long does it take to get set up?”
It can now be really easy.
More importantly than being easy, you are running an application’s tests within an isolated VM that you can design to match your production environment.
I think there can be a good future for CI and Rails applications. Thoughts on this solution?
Some of the technology in this article is old, other bits are very new, but for all of it I thank all the creators and contributors. Hudson CI was created by Kohsuke Kawaguchi during his days at Sun. He is now offering Hudson support services via his company InfraDNA.
The Hudson.rb project was created by Charles Lowell to bundle the Hudson CI and some common useful plugins for Ruby/Rails projects. He came all the way to Gothenburg, Sweden for NordicRuby conference (great conference by the way!) I fell in love with Hudson and started helping Charles on Hudson.rb.
Thanks also to both Kohsuke and Charles for starting work on a JRuby plugin for Hudson CI, and writing up a progress report.
Thanks to everyone who agrees that Hudson CI is awesome.
Thanks to Mitchell Hashimoto and John Bender for creating Vagrant. It is an incredible tool for developing within a VM (on OS X or Windows).
RubyConf & RubyBayou
If you’re in New Orleans this week for RubyConf, I’ll be at the local Ruby group RubyBayou talking about Hudson CI, Vagrant, and AppCloud on Thursday night. There’s a happy hour from 5pm till the 7pm start.
Come and let me convince you about the wonders of Hudson CI and having a CI test environment that matches your production env. You know you want to be convinced. It’s good for you, like fruit.
Don’t forget Chuck Norris
Always remember to install the Chuck Norris plugin for Hudson, and enable it for each job. Don’t forget Chuck Norris.