Dr Nic

Creating new BOSH releases with bosh-gen and bosh-solo

Below is a 45-minute walk-thru tutorial of creating and deploying a BOSH release for a Redis VM. It introduces a brand new tool: bosh-solo.

A BOSH release describes a complete running system from the ground up – compiled packages from source, templated configuration files, and monit to start/stop processes. BOSH itself then allows you to deploy your release across 1 or more VMs with optional persistent disks on the target infrastructure of your choice.

With bosh-solo you can develop and test BOSH releases without BOSH itself. Develop an entire working system quickly before you deploy to production.

“Just wanted to share a small personal achievement, last night I was able to
take a release I had created (actually, the redis-on-demand from your
tutorial) and bring it up using bosh-solo, connect to it, set/get, etc.

“I’ve gotta say, this is VERY slick. Awesome work on this.” – Brian McClain, creator of BoshDB.

Instant new Rails applications with the App Scrolls

When I start a new project I want to start how I plan to finish.

If I intend to write integration tests then I want to start them immediately. If I intend to set up continuous integration then I want CI setup immediately. If I intend to deploy the app somewhere then I want to deploy it immediately. If I intend to use some background workers, I want to know that they are setup immediately.

I want it all setup as easily as I can generate a new Rails app. They shouldn’t be extra work later. I’ll be busy then.

Assuming you want the same simplicity and speed, then let me share with you the App Scrolls!

gem install appscrolls
scrolls new mydemoapp -s twitter_bootstrap unicorn postgresql resque github eycloud

This one command will:

  • create a new Rails application called “mydemoapp”
  • install Twitter Bootstrap (you choose fixed or floating) and some other Rails basic tweaks
  • sets up Unicorn and Postgresql in the application
  • sets up Resque and Redis for background workers
  • hosts the project on GitHub (public or private repo)
  • deploys the application on Engine Yard Cloud

These are the magical App Scrolls!

Can you run the command above, or do you need a video to see it happen? Let me know.

Scrolls are interactive

$ scrolls new myapp -s twitter_bootstrap github resque postgresql unicorn eycloud
...
   question  Create a GitHub repository?
          1)  Public
          2)  Private
      github  Enter your selection: 1
...
      resque  Enter a secret string for the route /resque/YOUR-SECRET-STRING: mysecret
...
    question  Which Twitter Bootstrap layout?
          1)  Fluid
          2)  Fixed
  twitter_bootstrap  Enter your selection: 2
...
    question  Select application cluster configuration?
          1)  Basic - 1 app VM (5 CPU-based processes) & DB Master VM
          2)  Pro   - 3 app highly-available VMs (15 CPU-based processes) & DB Master VM
          3)  Solo  - 1 VM for app processes, DB and other services
     eycloud  Enter your selection: 1

Deploy automatically

I perfer to host on Engine Yard Cloud and I work there. So I added support to Engine Yard Cloud to all the scrolls and it will automatically boot a new environment and deploy your new application. Thanks to David Calavera’s unofficial ey_cli tool and the unofficial github-ruby CLI for making this possible!

Which Scrolls are supported?

Lots of magical scrolls are available!

administration: active_admin
assets: jquery, prototype
deployment: eycloud, eycloud_recipes_on_deploy, git, github, passenger, unicorn
persistence: mysql, postgresql, redis, sqlite3
stylesheet: twitter_bootstrap
templating: simple_form
testing: capybara, cucumber, rspec, test_unit
worker: delayed_job, resque
other: env_yaml, guard, rails_basics, split

I might blog about individual combinations later. My favourite is guard, which magically combines with other scrolls you choose.

Heroku or CloudFoundry?

If you’re a fan/employee of Salesforce/Heroku or VMWare/CloudFoundry or other platform, can I please ask for your help to make the current set of scrolls work? I’ll pair with you to get you started if you need it; though I think the code base and the scrolls are relatively self explanatory.

Isn’t this like Rails Wizard?

Yes! Before the App Scrolls, there was Michael Bleigh’s Rails Wizard. He created the first version during the 2010 Rails Rumble [Intridea blog post], and iterated on it a few times. I especially like that he extracted the project into two codebases – the CLI/scrolls and the web version.

I renamed it because ultimately I think it could support more than Rails applications and it definitely should become more than a “new application wizard”. And because I’ve been playing Elder Scrolls. Fortunately, Michael convinced me not to name the project after the computer game. Thanks to Thom Mahoney for suggesting “App Scrolls” and saving me from a naming nightmare.

A new name and a new repository? I’m still in two minds as to if I should have done that. Oh well, its done now. App Scrolls is a cool name.

Why “Scrolls”?

Rails Wizard used the word “recipe” to describe each component or service that would be added to a new application. Talking with others at Engine Yard, the word “recipe” got confusing. There were wizard recipes and chef recipes. When you wanted to use Resque you needed both. “Scrolls!” seemed a fun alternative name.

Wizards use scrolls, not recipes. Every D&D person knows that.

Where are all the old scrolls?

For the initial release, I’ve moved many contributed “recipes” from the Rails Wizard into the scrolls/zzz archive folder. There are so many and I don’t yet know how to ensure they all work as expected. I intend to find a good way to allow rubygems to self-host their scroll.

Continuous Integration?

I mentioned that I like CI to be setup immediately for new applications. That’s next. Jenkins, Travis CI (when they release private application support), and what other CI systems should be supported?

I hope you use the App Scrolls for your next big or small Rails app and that its useful to you!

Coming to America

Have you ever looked at your biological watch and thought, “it looks like it’s time to change the world?”

My biological time piece has a full set of inscriptions: go to university, chase girls, get a post-graduate qualification, catch a girl, get a professional job, marry the girl, change into contracting, change into consulting (and ponder what the difference is), work overseas (and marvel at the differences), make a baby, change into training, move back to Australia, buy a home, start a consultancy (Mocra), make another baby, grow the consultancy, and …

Really, I’m not sure what comes next in the script of normal life. Retire in 30 years? Create more little open source projects? Just keep growing the consultancy? Until what?

A few weeks ago, John Dillon, the CEO of Engine Yard, drew some pictures on a whiteboard for me and asked, “Do you want to help change the world?” Sure, a classic Steve Jobs one-liner from the history books of Apple. I said, “Yes”.

On the 1st of September I will start work at 500 3rd Street, San Francisco as Engine Yard
’s VP, Technology. Job description: help web application developers win.

22000km to work

The daily commute

Google Maps suggests my daily commute to work will be 22,300km. Some of that will be heavy morning traffic, so at say 30km/hr, that is almost 750 hours. Each way.

So instead, my family will pack its bags and move overseas again. Though for the first time, we’re coming to America!

How to move your family to America

It hasn’t been all smooth sailing on the home front since I broached the idea with my wife. Personally, have you ever turned to your wife and said, “Honey, I think we are going to move to America!”

Try it at home for a laugh. Perhaps your wife will say something witty or clever like mine. Something like, “No.”

“No, baby, really. I have something important to do for a few years. It’s in America. San Francisco. Apparently it’s lovely!”

“Is the weather nice?” she might ask looking for any reason to want to leave beautiful, sunny Brisbane.

“No. I think it’s kind of cold there. And foggy. There was even a taxi with ‘Fog City Taxi’ on its side. But there are wonderful schools and the people are fabulous and are from all over the world!”

“So our young kids won’t get an American accent?” she might ask pleadingly.

“Well, they won’t get all of the accents, no.”

“…”

“I love you!”

And so eventually your wife will say yes.

How do we get to win as web developers?

Choose to use Rails. Improve Rails. Improve the ecosystem above/below/around Rails. Improve our daily lives. Expand the sweet spot of Rails to solve problems.

In the future, can we look back at the work we did, the fun we had, the problems we solved, the time we spent working, and feel that it was worth the effort? Will we remember when we first discovered Ruby and Rails and remember the joy?

This is a world-changing set of issues to care about. I’ve always cared about them before now, but never had resources or reach to work on them beyond my own little projects or cunningly worded, suggestive emails/tweets/chat with other open source developers.

Though from now onwards I will get to work with one of the greatest concentrations of Rails talent and one of the largest open source programs in the our community. I’m more than just a little bit excited. Imagine working with full-time contributors to Rails, JRuby and Rubinius as well as the dozens of developers, devops and more.

As Engine Yard continues to grow and win then its capacity to fund and resource more contributions will grow. And then we all win faster.

Can you imagine what else should be funded, promoted or prioritised? I have a dozen solid ideas but I would love suggestions.

Mocra

I will miss my guys at Mocra, a wonderful Rails consultancy with clients around the world whilst based out of Brisbane, Australia. I founded and ran Mocra for the last two wonderful years. It will be slightly annoying – I won’t get to take any credit for all the success the Mocra team will have in the years to come.

Definitely, if you need an awesome Rails team for your project, contact Mocra. If you need a personal introduction, let me know. I know some people there.

San Francisco

I am very excited to move to SF and to hang out on a regular basis all the people I have only been able to see at conferences.

I’d also love suggestions of where a family of four (including a 4yo and 2yo; oh, plus another one due in February) could live. I’ve briefly seen parts of SF and many of the surrounding towns in the bay area. Thanks to Randall, Tammer and Marcy for their turns as tour guides during my visit a few weeks ago.

Customized Google Forms

Google Form Customized

Google Forms are a great, free way to collect information from anyone, stored directly into a Google Spreadsheet, and then have Google notify you each time a form is submitted (optionally). The downside is that you can only use one of their pre-packaged themes. You can’t have your company logo and corporate “look”.

Pooey to Google, we say. So we wrote custom_google_forms.

Want to truly customized Google Forms? Fork this repository, customize the CSS, DOM and images, and deploy to Heroku. You can then host/customize any number of Google Forms.

For example, here is an original bare Google Form and here is a fully customized version, including form validation and thank you page.

It’s fully a dynamic, thin layer on top of Google Forms. If you change your Google Form, your custom form application automatically changes.

How to use Google Forms?

Perhaps read this docco? Essentially if you can access Google Spreadsheets, you can create a Google Form (which automatically creates and updates a Google Spreadsheet based on the fields and form submissions).

Fork and Customize

It’s really simple to play with and customize Google Forms. Clone the GitHub project and fire up the Rails server:

gem install github
cd ~/Sites
gh clone mocra/custom-google-forms yourdomain-google-forms
cd yourdomain-google-forms
script/server

Go to http://localhost:3000/google_forms/new and add a Google Form:

My Google Forms - New

The “Slug” is the URL path people will follow. That is, a slug “railsdev” maps to http://forms.mocra.com/railsdev.

The “Form Key” field takes either the Form URL or just the formkey parameter (e.g. dFo0LXQyYmlEV2dXcVJ6WjRweW9vRnc6MA) from when you view the live form.

Press “Create” and follow the slug url to view your Google Form through your own customized style.

Well, initially you’ll see the Mocra style. Let’s fix that.

Default Theme

First, fork the github repository so that you can push your style changes to your own repo.

gh fork
mate public/stylesheets/style.css

Now, edit public/stylesheets/style.css to your tastes.

Please don’t reuse our style. Aside from you looking a bit silly having the :mocra logo at the top, it would be weird if your forms looked like our forms. I guess you might as well have used the default Google Form themes?

Heroku Deployment

Let’s not get fancy, let’s just deploy your fork to Heroku. It’s free. It’s simple. You’ll be done in two minutes.

To get your copy of the application deployed to heroku:

gem install heroku
heroku create yourdomain-google-forms
git push heroku master
heroku rake db:migrate

Two optional environment variables:

heroku config:add GOOGLE_ANALYTICS='YOURCODE'

If specified, Google Analytics will be included on all pages (including the custom Google Forms page).

heroku config:add EXPECTED_DOMAIN='some.yourdomain.com'

If specified, ‘yourdomain-google-forms.heroku.com/someform’ automatically redirects to ‘some.domain.com/someform’.

For example, for http://forms.mocra.com we used:

heroku config:add GOOGLE_ANALYTICS='UA-5370510-4'
heroku config:add EXPECTED_DOMAIN='forms.mocra.com'

Getting Started

Once deployed, go to the /google_forms/new URL and add your first form. Once it is validated and added, you can start using it using the links shown.

Summary

Yay for free, customized Google Forms! Yay for free Heroku hosting!

Bonus: you can configure your Google Forms/Spreadsheet to notify you when new forms are submitted. That’s handy.

Thanks to Odin Dutton, our resident designer at Mocra, who did a great job applying our new theme to the Google Forms DOM structure. It’s wonderful having him on the team!

First 5 Minutes of Stand Up Comedy

I thought you might be interested to see the fruits of my new hobby: stand up comedy.

It should be running in “HD” for 20% more laughs. Video taken by Greg Fairbrother, my awesome Mocra Off Railers co-driver.

I discovered a local Stand Up Comedy Course which started 6 weeks ago. The final “night” of the course was for the 8 of us to do a show for family and friends. I don’t remember reading about that on the sales brochure.

Nonetheless, knowing you had a 5 minute set to perform in front of your family and friends sets the expectation in your mind from day one that you don’t want to be shit. So we listened up, wrote as many jokes as we could think of, and hoped desperately we wouldn’t be a blubbering mess on the night.

It’s not all left to chance. Each week, the funny woman who ran the course, Fiona McGary, made us use the microphone on the stage. Either talk about our week, or list all the funny things you can think of about frogs. Or any other European people.

Each week we boldly attempted to make our other classmates laugh. At the start, they don’t. So I quickly learned to edit and then how to present comedy on stage. I tried to learn it fast because it’s weird telling a punch line, waiting for the laughs, and only being rewarded with a uncomfortable shared silence.

All my jokes started out crappy. So I attempted to edit them into “setup-punchline format”. Another phrase for “edit” is “delete half”. I’m a verbose writer. This turns out to suck awfully for stand up comedy. Or perhaps I should be a concise writer too. Oh the novelty of the idea.

Another way to edit a joke so it is ready for an audience is to delete all of it. Turns out, just because I thought of something funny doesn’t mean anyone else thinks it’s funny. Ewwww, that was an uncomfortable life lesson.

I don’t know anything else and the above is probably wrong too. The last six weeks have been a blur. Lots of practise, lots of editing, lots of writing, lots of testing ideas on Mocra staff when they aren’t expecting it, and weekly doses of disturbing reality at the Tuesday workshops when I realise I still have 10 years of more practise to go before David Letterman might ask me to come on his show. He’ll only be 112 years old by then, so fingers crossed.

I’m very appreciative of the 20+ family and friends who came along for the show. All the comedians were very appreciative of especially Lucas and Chendo who sat at the front and giggled like little girls all night.

Spending the last six weeks with the other noob comedians has been wonderful. It will be fun seeing everyone around the Open Mic rooms in Brisbane.

Finally, and most importantly, thanks to Fiona for running the course, and bumping us forward each week with the subtle phrase “that’s good; though it will need more editing.”