Dr Nic

Creating a BOSH from scratch on AWS

BOSH architecture

A lot of devops projects revolve around managing instances/VMs once they already exist. For example, Chef and Puppet do configuration management of instances once the instances have been created. The “monitoring sucks” work is concerned with monitoring the contents/jobs/processes of instances and the instances themselves. Libraries such as fog and jclouds dedicate themselves to provisioning instances and other cloud resources.

BOSH attempts to do something that I haven’t seen in OSS yet. It wants to allow a devops team to describe the entire software stack – from the base operating system image used to boot new instances (stemcell), to the software packages installed on instances (packages), to the processes that are run on each instance (jobs). It wants to allow a devops team to describe the runtime deployment of your stack – which instance types are used, how many of them, how much disk is attached, and how networking properties such as IPs are configured.

More than that, BOSH then wants to allow you to change the configuration – change instance types (scale vertically), how many of them (scale horizontally), upgrade packages, and even upgrade the base image – and deploy all those changes/deltas to the running system.

That’s what it wants to do.

And then I saw a demo of it doing it. This demo was running against vSphere. There was a running deployment of instances, which I could also see in vCenter. A simple YAML configuration file was changed – he changed the instance specification to add more CPUs to each instance – and ran bosh deploy.

Then I watched vCenter light up as it reflectively showed that disks were being detached; instances were being torn down; new, bigger CPU instances being booted, and the disks reattached.

I needed one of these BOSH things immediately. It has some good high-level and low-level documentation

But first, I needed a BOSH. And I needed a BOSH that could talk to AWS instead of vSphere.

Below is a screencast of creating your own BOSH on AWS. The written instructions are also available.

Thanks to a whole bunch of CloudFoundry folk who wrote BOSH and also helped me figure out how to set it up: Oleg, Vadim, skaar, Martin, and Mahesh. Also to Mark, Dave, Matt and James for sharing and caring when I visited the team.

What’s next? How do I use BOSH?

Once you get a BOSH created, I’ve started some other tutorials for how to use BOSH as I figure it out. Perhaps it could become a handbook or guide in the future. If you find any bugs, or have suggestions, please drop a ticket in the Issues.

At the time of writing, there are the following tutorials/guides:

Related posts:

  1. Creating new BOSH releases with bosh-gen and bosh-solo Below is a 45-minute walk-thru tutorial of creating and deploying...
  2. BOSH: What, How, When I’m still very bullish on BOSH. I’ve been experimenting with...

2 Responses to “Creating a BOSH from scratch on AWS”

  1. [...] Dr Nic’s Creating a BOSH from scratch on AWS – from Dr Nic http://drnicwilliams.com [...]

  2. [...] Cloud Foundry BOSH including the work being done by Nic Williams of Engine Yard. See his recent blog on his experiences with BOSH, as well as his public talk this Wednesday (May 2, 2012).  Join in on [...]