Dr Nic

Instant new Rails applications with the App Scrolls

When I start a new project I want to start how I plan to finish.

If I intend to write integration tests then I want to start them immediately. If I intend to set up continuous integration then I want CI setup immediately. If I intend to deploy the app somewhere then I want to deploy it immediately. If I intend to use some background workers, I want to know that they are setup immediately.

I want it all setup as easily as I can generate a new Rails app. They shouldn’t be extra work later. I’ll be busy then.

Assuming you want the same simplicity and speed, then let me share with you the App Scrolls!

gem install appscrolls
scrolls new mydemoapp -s twitter_bootstrap unicorn postgresql resque github eycloud

This one command will:

  • create a new Rails application called “mydemoapp”
  • install Twitter Bootstrap (you choose fixed or floating) and some other Rails basic tweaks
  • sets up Unicorn and Postgresql in the application
  • sets up Resque and Redis for background workers
  • hosts the project on GitHub (public or private repo)
  • deploys the application on Engine Yard Cloud

These are the magical App Scrolls!

Can you run the command above, or do you need a video to see it happen? Let me know.

Scrolls are interactive

$ scrolls new myapp -s twitter_bootstrap github resque postgresql unicorn eycloud
...
   question  Create a GitHub repository?
          1)  Public
          2)  Private
      github  Enter your selection: 1
...
      resque  Enter a secret string for the route /resque/YOUR-SECRET-STRING: mysecret
...
    question  Which Twitter Bootstrap layout?
          1)  Fluid
          2)  Fixed
  twitter_bootstrap  Enter your selection: 2
...
    question  Select application cluster configuration?
          1)  Basic - 1 app VM (5 CPU-based processes) & DB Master VM
          2)  Pro   - 3 app highly-available VMs (15 CPU-based processes) & DB Master VM
          3)  Solo  - 1 VM for app processes, DB and other services
     eycloud  Enter your selection: 1

Deploy automatically

I perfer to host on Engine Yard Cloud and I work there. So I added support to Engine Yard Cloud to all the scrolls and it will automatically boot a new environment and deploy your new application. Thanks to David Calavera’s unofficial ey_cli tool and the unofficial github-ruby CLI for making this possible!

Which Scrolls are supported?

Lots of magical scrolls are available!

administration: active_admin
assets: jquery, prototype
deployment: eycloud, eycloud_recipes_on_deploy, git, github, passenger, unicorn
persistence: mysql, postgresql, redis, sqlite3
stylesheet: twitter_bootstrap
templating: simple_form
testing: capybara, cucumber, rspec, test_unit
worker: delayed_job, resque
other: env_yaml, guard, rails_basics, split

I might blog about individual combinations later. My favourite is guard, which magically combines with other scrolls you choose.

Heroku or CloudFoundry?

If you’re a fan/employee of Salesforce/Heroku or VMWare/CloudFoundry or other platform, can I please ask for your help to make the current set of scrolls work? I’ll pair with you to get you started if you need it; though I think the code base and the scrolls are relatively self explanatory.

Isn’t this like Rails Wizard?

Yes! Before the App Scrolls, there was Michael Bleigh’s Rails Wizard. He created the first version during the 2010 Rails Rumble [Intridea blog post], and iterated on it a few times. I especially like that he extracted the project into two codebases – the CLI/scrolls and the web version.

I renamed it because ultimately I think it could support more than Rails applications and it definitely should become more than a “new application wizard”. And because I’ve been playing Elder Scrolls. Fortunately, Michael convinced me not to name the project after the computer game. Thanks to Thom Mahoney for suggesting “App Scrolls” and saving me from a naming nightmare.

A new name and a new repository? I’m still in two minds as to if I should have done that. Oh well, its done now. App Scrolls is a cool name.

Why “Scrolls”?

Rails Wizard used the word “recipe” to describe each component or service that would be added to a new application. Talking with others at Engine Yard, the word “recipe” got confusing. There were wizard recipes and chef recipes. When you wanted to use Resque you needed both. “Scrolls!” seemed a fun alternative name.

Wizards use scrolls, not recipes. Every D&D person knows that.

Where are all the old scrolls?

For the initial release, I’ve moved many contributed “recipes” from the Rails Wizard into the scrolls/zzz archive folder. There are so many and I don’t yet know how to ensure they all work as expected. I intend to find a good way to allow rubygems to self-host their scroll.

Continuous Integration?

I mentioned that I like CI to be setup immediately for new applications. That’s next. Jenkins, Travis CI (when they release private application support), and what other CI systems should be supported?

I hope you use the App Scrolls for your next big or small Rails app and that its useful to you!

Coming to America

Have you ever looked at your biological watch and thought, “it looks like it’s time to change the world?”

My biological time piece has a full set of inscriptions: go to university, chase girls, get a post-graduate qualification, catch a girl, get a professional job, marry the girl, change into contracting, change into consulting (and ponder what the difference is), work overseas (and marvel at the differences), make a baby, change into training, move back to Australia, buy a home, start a consultancy (Mocra), make another baby, grow the consultancy, and …

Really, I’m not sure what comes next in the script of normal life. Retire in 30 years? Create more little open source projects? Just keep growing the consultancy? Until what?

A few weeks ago, John Dillon, the CEO of Engine Yard, drew some pictures on a whiteboard for me and asked, “Do you want to help change the world?” Sure, a classic Steve Jobs one-liner from the history books of Apple. I said, “Yes”.

On the 1st of September I will start work at 500 3rd Street, San Francisco as Engine Yard
’s VP, Technology. Job description: help web application developers win.

22000km to work

The daily commute

Google Maps suggests my daily commute to work will be 22,300km. Some of that will be heavy morning traffic, so at say 30km/hr, that is almost 750 hours. Each way.

So instead, my family will pack its bags and move overseas again. Though for the first time, we’re coming to America!

How to move your family to America

It hasn’t been all smooth sailing on the home front since I broached the idea with my wife. Personally, have you ever turned to your wife and said, “Honey, I think we are going to move to America!”

Try it at home for a laugh. Perhaps your wife will say something witty or clever like mine. Something like, “No.”

“No, baby, really. I have something important to do for a few years. It’s in America. San Francisco. Apparently it’s lovely!”

“Is the weather nice?” she might ask looking for any reason to want to leave beautiful, sunny Brisbane.

“No. I think it’s kind of cold there. And foggy. There was even a taxi with ‘Fog City Taxi’ on its side. But there are wonderful schools and the people are fabulous and are from all over the world!”

“So our young kids won’t get an American accent?” she might ask pleadingly.

“Well, they won’t get all of the accents, no.”

“…”

“I love you!”

And so eventually your wife will say yes.

How do we get to win as web developers?

Choose to use Rails. Improve Rails. Improve the ecosystem above/below/around Rails. Improve our daily lives. Expand the sweet spot of Rails to solve problems.

In the future, can we look back at the work we did, the fun we had, the problems we solved, the time we spent working, and feel that it was worth the effort? Will we remember when we first discovered Ruby and Rails and remember the joy?

This is a world-changing set of issues to care about. I’ve always cared about them before now, but never had resources or reach to work on them beyond my own little projects or cunningly worded, suggestive emails/tweets/chat with other open source developers.

Though from now onwards I will get to work with one of the greatest concentrations of Rails talent and one of the largest open source programs in the our community. I’m more than just a little bit excited. Imagine working with full-time contributors to Rails, JRuby and Rubinius as well as the dozens of developers, devops and more.

As Engine Yard continues to grow and win then its capacity to fund and resource more contributions will grow. And then we all win faster.

Can you imagine what else should be funded, promoted or prioritised? I have a dozen solid ideas but I would love suggestions.

Mocra

I will miss my guys at Mocra, a wonderful Rails consultancy with clients around the world whilst based out of Brisbane, Australia. I founded and ran Mocra for the last two wonderful years. It will be slightly annoying – I won’t get to take any credit for all the success the Mocra team will have in the years to come.

Definitely, if you need an awesome Rails team for your project, contact Mocra. If you need a personal introduction, let me know. I know some people there.

San Francisco

I am very excited to move to SF and to hang out on a regular basis all the people I have only been able to see at conferences.

I’d also love suggestions of where a family of four (including a 4yo and 2yo; oh, plus another one due in February) could live. I’ve briefly seen parts of SF and many of the surrounding towns in the bay area. Thanks to Randall, Tammer and Marcy for their turns as tour guides during my visit a few weeks ago.

Packaging TextMate bundles in OS X DMGs

Last week Engine Yard released a CLI for their Engine Yard AppCloud. Delights such as:

ey deploy
ey rebuild
ey logs
ey ssh

Engine Yard.tmbundle

They simultaneously released a TextMate bundle to deploy, rebuild, view logs, etc using Ctrl+Alt+Cmd+E. Like all TextMate bundles, you can install it in one of two ways: via git (see the README), or via a beautiful DMG. Download it here!

Yes indeed, TextMate bundles can now be packaged up and distributed via DMGs using ChocTop!

Ruby on Rails.tmbundle

For example, the Ruby on Rails.tmbundle using a simple purple theme.

First, the Engine Yard tmbundle

To use the Engine Yard tmbundle, you first need to install and use the CLI once. Instructions at the bottom of the information page.

How to package a TextMate bundle into a DMG

ChocTop is a packaging and distribution tool originally designed only for Cocoa applications, but can now package any assets, URL links, or even the whole project folder itself. This makes it ideal for packaging TextMate bundles which have no compiled/built output to distribute (like a Cocoa application), rather the project folder itself is the distributed item (the Engine Yard.tmbundle folder in this case).

Getting started

Everything is added into your TextMate bundle project. For example, with the EngineYard bundle:

gem install choctop
cd Library/Application\ Support/TextMate/Bundles/Engine\ Yard.tmbundle
install_choctop . --tmbundle

If your tmbundle already has a Rakefile, then don’t overwrite it. Instead, inside the Rakefile, add the ChocTop configuration:

require "choctop"

ChocTop::Configuration.new do |s|
  s.add_root :position => [290, 200], :exclude => %w[appcast build .bundle .git]
  s.add_link 'http://github.com/engineyard/engineyard.tmbundle', 'GitHub', :position => [520, 200]
  s.defaults :textmate
end

For TextMate bundles the DMG magic is from the s.add_root line. The resulting DMG will include the entire project as a folder/bundle. For example, you’ll want to exclude appcast, build, .bundle (if you’re using Bundler), and .git folders.

The s.defaults :textmate provides a generic background and volume icon for a TextMate bundle DMG. See below for customising the background and volume icons. The :position coordinates above are for the generic background.

Building your DMG

To build your DMG and then view it in Finder:

rake dmg
open appcast/build/*.dmg
# or together
rake dmg[automount]

You can now share the DMG file. See below for how to upload it to a server.

Versioning

In future, it would be great to use Sparkle’s auto-update mechanism (as seen in nearly every Cocoa application). ChocTop will automatically generate the required XML feed; TextMate nor the bundle has a way to ask Sparkle to poll for it nor update itself, yet.

But, you can start versioning your DMGs today:

$ rake version:current
0.0.0
$ rake version:bump:major
$ rake version:current
1.0.0
$ rake dmg

The DMG will now have a version number.

Uploading new DMG versions

The original ChocTop was designed for Cocoa applications and included Sparkle support so your Cocoa applications automatically updated themselves when you built and uploaded a new version. I haven’t got a solution for this for TextMate bundles yet; but it seems like a good idea for the future.

Nonetheless, ChocTop still includes a rake upload task to ship new versions of your DMG to a server somewhere.

In your Rakefile, add the following config lines to the ChocTop block:

s.base_url   = 'http://some.host.com/upload/folder'
s.remote_dir = '/path/to/upload/folder'
s.host       = 'some.host.com'
s.user       = 'remote-user'

The s.base_url is the URL from where the DMG file will be found by users. Later, when I figure out how to do auto-updating of the TextMate bundles, it will also use this URL. This URL is also used to determine the host for uploading the file.

The s.remote_dir is a path on the target server that maps to the base_url. This folder must already exist; rsync cannot create it as far as I can tell. So ssh into the machine and mkdir -p /path/to/upload/folder

The latter two are optional: s.host is derived from s.base_url and s.user defaults to your current local user.

To upload the latest DMG, run:

rake upload

You can now share the URL http://some.host.com/upload/folder for people to download the DMG. A small PHP script redirects from the folder path to the DMG filename.

Customising

ChocTop allows you to customise nearly everything.

The s.defaults :textmate line is similar to the following configuration:

s.background_file = "...choctop/assets/textmate_background.jpg"
s.volume_icon     = "...choctop/assets/textmate_volume.icns"
s.icon_size       = 104
s.icon_text_size  = 12

For TextMate bundles, perhaps put customised assets into a Support/dmg folder.

A background image should include blank space for the large YourBundle.tmbundle icon and webloc URL file to your GitHub project (or other target URLs). There are no size constraints on the background image. Design something beautiful.

The volume icon is an icns file. You create this using OS X’s Icon Composer application. Start with a transparent png file and drop it into the box with the corresponding size.

For the Engine Yard tmbundle, the following configuration is used:

s.background_file = 'Support/dmg/engineyard.tmbundle.dmg.png'
s.volume_icon     = 'Support/dmg/engineyard.dmg.icns'

Additional files

If there are other files you explicitly want bundled in the DMG, say a pretty README.html or a folder of documentation, then you can specify them:

s.add_file 'README.html', :position => [50, 100]
s.add_file 'docs', :position => [100, 100], :name => 'Documentation'

Summary

ChocTop is pretty cool for bundling any set of files into a custom DMG, especially Cocoa applications and now TextMate bundles.

Hopefully one day we can have Sparkle auto-updates for TextMate bundles too.