Dr Nic

Creating new BOSH releases with bosh-gen and bosh-solo

Below is a 45-minute walk-thru tutorial of creating and deploying a BOSH release for a Redis VM. It introduces a brand new tool: bosh-solo.

A BOSH release describes a complete running system from the ground up – compiled packages from source, templated configuration files, and monit to start/stop processes. BOSH itself then allows you to deploy your release across 1 or more VMs with optional persistent disks on the target infrastructure of your choice.

With bosh-solo you can develop and test BOSH releases without BOSH itself. Develop an entire working system quickly before you deploy to production.

“Just wanted to share a small personal achievement, last night I was able to
take a release I had created (actually, the redis-on-demand from your
tutorial) and bring it up using bosh-solo, connect to it, set/get, etc.

“I’ve gotta say, this is VERY slick. Awesome work on this.” – Brian McClain, creator of BoshDB.

Creating a BOSH from scratch on AWS

BOSH architecture

A lot of devops projects revolve around managing instances/VMs once they already exist. For example, Chef and Puppet do configuration management of instances once the instances have been created. The “monitoring sucks” work is concerned with monitoring the contents/jobs/processes of instances and the instances themselves. Libraries such as fog and jclouds dedicate themselves to provisioning instances and other cloud resources.

BOSH attempts to do something that I haven’t seen in OSS yet. It wants to allow a devops team to describe the entire software stack – from the base operating system image used to boot new instances (stemcell), to the software packages installed on instances (packages), to the processes that are run on each instance (jobs). It wants to allow a devops team to describe the runtime deployment of your stack – which instance types are used, how many of them, how much disk is attached, and how networking properties such as IPs are configured.

More than that, BOSH then wants to allow you to change the configuration – change instance types (scale vertically), how many of them (scale horizontally), upgrade packages, and even upgrade the base image – and deploy all those changes/deltas to the running system.

That’s what it wants to do.

And then I saw a demo of it doing it. This demo was running against vSphere. There was a running deployment of instances, which I could also see in vCenter. A simple YAML configuration file was changed – he changed the instance specification to add more CPUs to each instance – and ran bosh deploy.

Then I watched vCenter light up as it reflectively showed that disks were being detached; instances were being torn down; new, bigger CPU instances being booted, and the disks reattached.

I needed one of these BOSH things immediately. It has some good high-level and low-level documentation

But first, I needed a BOSH. And I needed a BOSH that could talk to AWS instead of vSphere.

Below is a screencast of creating your own BOSH on AWS. The written instructions are also available.

Thanks to a whole bunch of CloudFoundry folk who wrote BOSH and also helped me figure out how to set it up: Oleg, Vadim, skaar, Martin, and Mahesh. Also to Mark, Dave, Matt and James for sharing and caring when I visited the team.

What’s next? How do I use BOSH?

Once you get a BOSH created, I’ve started some other tutorials for how to use BOSH as I figure it out. Perhaps it could become a handbook or guide in the future. If you find any bugs, or have suggestions, please drop a ticket in the Issues.

At the time of writing, there are the following tutorials/guides:

Instant new Rails applications with the App Scrolls

When I start a new project I want to start how I plan to finish.

If I intend to write integration tests then I want to start them immediately. If I intend to set up continuous integration then I want CI setup immediately. If I intend to deploy the app somewhere then I want to deploy it immediately. If I intend to use some background workers, I want to know that they are setup immediately.

I want it all setup as easily as I can generate a new Rails app. They shouldn’t be extra work later. I’ll be busy then.

Assuming you want the same simplicity and speed, then let me share with you the App Scrolls!

gem install appscrolls
scrolls new mydemoapp -s twitter_bootstrap unicorn postgresql resque github eycloud

This one command will:

  • create a new Rails application called “mydemoapp”
  • install Twitter Bootstrap (you choose fixed or floating) and some other Rails basic tweaks
  • sets up Unicorn and Postgresql in the application
  • sets up Resque and Redis for background workers
  • hosts the project on GitHub (public or private repo)
  • deploys the application on Engine Yard Cloud

These are the magical App Scrolls!

Can you run the command above, or do you need a video to see it happen? Let me know.

Scrolls are interactive

$ scrolls new myapp -s twitter_bootstrap github resque postgresql unicorn eycloud
   question  Create a GitHub repository?
          1)  Public
          2)  Private
      github  Enter your selection: 1
      resque  Enter a secret string for the route /resque/YOUR-SECRET-STRING: mysecret
    question  Which Twitter Bootstrap layout?
          1)  Fluid
          2)  Fixed
  twitter_bootstrap  Enter your selection: 2
    question  Select application cluster configuration?
          1)  Basic - 1 app VM (5 CPU-based processes) & DB Master VM
          2)  Pro   - 3 app highly-available VMs (15 CPU-based processes) & DB Master VM
          3)  Solo  - 1 VM for app processes, DB and other services
     eycloud  Enter your selection: 1

Deploy automatically

I perfer to host on Engine Yard Cloud and I work there. So I added support to Engine Yard Cloud to all the scrolls and it will automatically boot a new environment and deploy your new application. Thanks to David Calavera’s unofficial ey_cli tool and the unofficial github-ruby CLI for making this possible!

Which Scrolls are supported?

Lots of magical scrolls are available!

administration: active_admin
assets: jquery, prototype
deployment: eycloud, eycloud_recipes_on_deploy, git, github, passenger, unicorn
persistence: mysql, postgresql, redis, sqlite3
stylesheet: twitter_bootstrap
templating: simple_form
testing: capybara, cucumber, rspec, test_unit
worker: delayed_job, resque
other: env_yaml, guard, rails_basics, split

I might blog about individual combinations later. My favourite is guard, which magically combines with other scrolls you choose.

Heroku or CloudFoundry?

If you’re a fan/employee of Salesforce/Heroku or VMWare/CloudFoundry or other platform, can I please ask for your help to make the current set of scrolls work? I’ll pair with you to get you started if you need it; though I think the code base and the scrolls are relatively self explanatory.

Isn’t this like Rails Wizard?

Yes! Before the App Scrolls, there was Michael Bleigh’s Rails Wizard. He created the first version during the 2010 Rails Rumble [Intridea blog post], and iterated on it a few times. I especially like that he extracted the project into two codebases – the CLI/scrolls and the web version.

I renamed it because ultimately I think it could support more than Rails applications and it definitely should become more than a “new application wizard”. And because I’ve been playing Elder Scrolls. Fortunately, Michael convinced me not to name the project after the computer game. Thanks to Thom Mahoney for suggesting “App Scrolls” and saving me from a naming nightmare.

A new name and a new repository? I’m still in two minds as to if I should have done that. Oh well, its done now. App Scrolls is a cool name.

Why “Scrolls”?

Rails Wizard used the word “recipe” to describe each component or service that would be added to a new application. Talking with others at Engine Yard, the word “recipe” got confusing. There were wizard recipes and chef recipes. When you wanted to use Resque you needed both. “Scrolls!” seemed a fun alternative name.

Wizards use scrolls, not recipes. Every D&D person knows that.

Where are all the old scrolls?

For the initial release, I’ve moved many contributed “recipes” from the Rails Wizard into the scrolls/zzz archive folder. There are so many and I don’t yet know how to ensure they all work as expected. I intend to find a good way to allow rubygems to self-host their scroll.

Continuous Integration?

I mentioned that I like CI to be setup immediately for new applications. That’s next. Jenkins, Travis CI (when they release private application support), and what other CI systems should be supported?

I hope you use the App Scrolls for your next big or small Rails app and that its useful to you!

Making CI easier to do than not to with Hudson CI and Vagrant

It irked me a little that I could develop on one stack (OS X, Rubinius, Sqlite3), run continuous integration (CI) on another stack (Ubuntu, Ruby 1.8.7, Postgresql), and deploy into another stack (Gentoo, Ruby 1.9.2, MySQL). I think what irks and worries me is that there are three sets of differences to be aware of. A bug in production? Was it a missing test scenario or one of the many differences between production and CI environments?

So I think I have two solutions.

First, use a VM that matches the production environment. Each different production environment would mean another VM. If you are managing your own production environment, then all you need is the tools (described in this article) to recreate your production environment in a VM.

Second, use a clone of your production environment. That is, if you deploy to Engine Yard AppCloud then run CI in Engine Yard AppCloud; if you deploy to a single Ubuntu instance on Slicehost, the have another matching Ubuntu instance on Slicehost.

I’ll write about the first solution – using VMs – here, and I’ll write on the Engine Yard Blog about the solution for Engine Yard AppCloud customers. For AppCloud users life will be even easier because there are zero setup steps to ensure you have consistent environments. It’s been one of my favorite projects in the two months since I arrived at Engine Yard.

I’ll also introduce a CLI for talking to Hudson – one that assumes you are working on Ruby/Rails projects – and makes it really easy to get up and running with a server, your Rails/Ruby projects, and any VMs you need (optional for Hudson, but they are the point of my line of thinking).

And I’ll introduce Vagrant, a CLI for creating/managing/destroying VMs.

Oh, and I’ll introduce Hudson CI.

Ok, let’s fix all our CI problems in one go…

Hudson CI

I needed a CI tool to allow tests to run inside VMs or on remote servers/VMs. Back in May I found and fell in love with Hudson CI. Fortunately it had support for “slaves”, and a way for CI jobs (your applications or rubygems) to select which slaves can be used. Hudson is also great because it is easy to try out (one click to install and launch), easy to configure, and has 350+ plugins.

What is Hudson?

CLI for Hudson

Charles Lowell’s hudson.rb project is a CLI for launching Hudson (it’s bundled inside the gem), and a set of CLI tasks to add/remove projects (jobs), add slaves, trigger builds and more.

gem install hudson

You can launch Hudson via:

hudson server  # Default: --port 3001
open http://localhost:3001

It currently assumes you are working on Ruby projects, using bundler, and attempts to create a useful set of default steps for your CI jobs.

cd /some/rails3/app
hudson create . --host localhost --port 3001 --template rails3

Hudson CI will automatically start running steps to install your application’s gems, database, and run your tests. When it does it on your local machine it’s not that impressive. When it happens on a fresh VM or slave node and you didn’t have to do anything to set it up, it’s awesome.

V is for Vagrant, VirtualBox and all things VM

If you can script it, then you can automate it. Fortunately, as a virtual machine VirtualBox is both scriptable and FREE! Secondly, if you have a VM, you’ll need to script its setup/provisioning; so fortunately there exists chef and puppet, amongst others.

Thirdly, if you are really really really lazy, like me, you will want Vagrant. Created by Mitchell Hashimoto and John Bender, Vagrant is a tool for building and distributing virtualized development environments. Everyone in your team have their own OS/configuration? Got Windows users on your team? Use Vagrant.

Once you have VirtualBox installed, getting any of your projects live within a VM is trival:

gem install vagrant
vagrant box add base http://files.vagrantup.com/lucid32.box
vagrant init
vagrant up
vagrant ssh
$ cd /vagrant/
$ ls -al

You will now see the contents of your project folder, but from within the VM instance! They are linked – a change to either is automatically reflected inside and outside of the VM.

In this example, the Guest OS being downloaded and installed into VirtualBox (note: without any GUI) is Ubuntu Lucid, but you could use any VirtualBox packaged unix system.

For wonderful guided tour of Vagrant see the Getting Started video.

Vagrant – Getting Started from Mitchell Hashimoto on Vimeo.

Hudson CI + Vagrant = Perfect CI

Whether you choose to use VMs for a production-like development environment (a good idea for anyone, a wonderful idea for Windows developers who are very far removed from their production experience), it is a very good idea to have a CI environment as similar to your production environment as possible. Here, we want a VM instance with all the components/utilities/rubies set up that you have in production.

I have created an example Rails application that is setup to use Vagrant for a CI slave VM.

See the README for complete instructions. See the Vagrantfile and the cookbooks folder for the configuration and provisioning recipes.

Once you have the VM instantiated, you add it to your Hudson CI master (either the localhost one above or your remotely hosted server) with the CLI:

hudson add_node localhost --name "VM" --label railsapp-vagrant ...

(See the example Rails application for the other flags I used to get this working). It is now available to all Hudson jobs; but has a label “railsapp-vagrant” to allow jobs (Hudson’s name for a project) to specify that slave node specifically.

To add the Rails application to Hudson CI, and force it to run the tests in this VM:

hudson create . --template rails3 --assigned-node railsapp-vagrant

If you visit the Hudson master (at http://localhost:3010 in the example) you will see the job automatically running (“building”) within your VM. It will use bundler to install the gems, and run all the tests.

That’s it?

I know, CI is historically a pain in the arse. When CruiseControl was the only CI kid-on-the-block, it was standard for people to respond “4 days” to the question “How long does it take to get set up?”

It can now be really easy.

More importantly than being easy, you are running an application’s tests within an isolated VM that you can design to match your production environment.

I think there can be a good future for CI and Rails applications. Thoughts on this solution?


Some of the technology in this article is old, other bits are very new, but for all of it I thank all the creators and contributors. Hudson CI was created by Kohsuke Kawaguchi during his days at Sun. He is now offering Hudson support services via his company InfraDNA.

The Hudson.rb project was created by Charles Lowell to bundle the Hudson CI and some common useful plugins for Ruby/Rails projects. He came all the way to Gothenburg, Sweden for NordicRuby conference (great conference by the way!) I fell in love with Hudson and started helping Charles on Hudson.rb.

Thanks also to both Kohsuke and Charles for starting work on a JRuby plugin for Hudson CI, and writing up a progress report.

Thanks to everyone who agrees that Hudson CI is awesome.

Thanks to Mitchell Hashimoto and John Bender for creating Vagrant. It is an incredible tool for developing within a VM (on OS X or Windows).

Finally, thanks to Bo Jeanes who helped on the Engine Yard AppCloud version of this project. Coming soon!

RubyConf & RubyBayou

If you’re in New Orleans this week for RubyConf, I’ll be at the local Ruby group RubyBayou talking about Hudson CI, Vagrant, and AppCloud on Thursday night. There’s a happy hour from 5pm till the 7pm start.

Location: LaunchPad NOLA, 643 Magazine St, New Orleans, LA 70130

Come and let me convince you about the wonders of Hudson CI and having a CI test environment that matches your production env. You know you want to be convinced. It’s good for you, like fruit.

Don’t forget Chuck Norris

Always remember to install the Chuck Norris plugin for Hudson, and enable it for each job. Don’t forget Chuck Norris.