Dr Nic

Instant new Rails applications with the App Scrolls

When I start a new project I want to start how I plan to finish.

If I intend to write integration tests then I want to start them immediately. If I intend to set up continuous integration then I want CI setup immediately. If I intend to deploy the app somewhere then I want to deploy it immediately. If I intend to use some background workers, I want to know that they are setup immediately.

I want it all setup as easily as I can generate a new Rails app. They shouldn’t be extra work later. I’ll be busy then.

Assuming you want the same simplicity and speed, then let me share with you the App Scrolls!

gem install appscrolls
scrolls new mydemoapp -s twitter_bootstrap unicorn postgresql resque github eycloud

This one command will:

  • create a new Rails application called “mydemoapp”
  • install Twitter Bootstrap (you choose fixed or floating) and some other Rails basic tweaks
  • sets up Unicorn and Postgresql in the application
  • sets up Resque and Redis for background workers
  • hosts the project on GitHub (public or private repo)
  • deploys the application on Engine Yard Cloud

These are the magical App Scrolls!

Can you run the command above, or do you need a video to see it happen? Let me know.

Scrolls are interactive

$ scrolls new myapp -s twitter_bootstrap github resque postgresql unicorn eycloud
...
   question  Create a GitHub repository?
          1)  Public
          2)  Private
      github  Enter your selection: 1
...
      resque  Enter a secret string for the route /resque/YOUR-SECRET-STRING: mysecret
...
    question  Which Twitter Bootstrap layout?
          1)  Fluid
          2)  Fixed
  twitter_bootstrap  Enter your selection: 2
...
    question  Select application cluster configuration?
          1)  Basic - 1 app VM (5 CPU-based processes) & DB Master VM
          2)  Pro   - 3 app highly-available VMs (15 CPU-based processes) & DB Master VM
          3)  Solo  - 1 VM for app processes, DB and other services
     eycloud  Enter your selection: 1

Deploy automatically

I perfer to host on Engine Yard Cloud and I work there. So I added support to Engine Yard Cloud to all the scrolls and it will automatically boot a new environment and deploy your new application. Thanks to David Calavera’s unofficial ey_cli tool and the unofficial github-ruby CLI for making this possible!

Which Scrolls are supported?

Lots of magical scrolls are available!

administration: active_admin
assets: jquery, prototype
deployment: eycloud, eycloud_recipes_on_deploy, git, github, passenger, unicorn
persistence: mysql, postgresql, redis, sqlite3
stylesheet: twitter_bootstrap
templating: simple_form
testing: capybara, cucumber, rspec, test_unit
worker: delayed_job, resque
other: env_yaml, guard, rails_basics, split

I might blog about individual combinations later. My favourite is guard, which magically combines with other scrolls you choose.

Heroku or CloudFoundry?

If you’re a fan/employee of Salesforce/Heroku or VMWare/CloudFoundry or other platform, can I please ask for your help to make the current set of scrolls work? I’ll pair with you to get you started if you need it; though I think the code base and the scrolls are relatively self explanatory.

Isn’t this like Rails Wizard?

Yes! Before the App Scrolls, there was Michael Bleigh’s Rails Wizard. He created the first version during the 2010 Rails Rumble [Intridea blog post], and iterated on it a few times. I especially like that he extracted the project into two codebases – the CLI/scrolls and the web version.

I renamed it because ultimately I think it could support more than Rails applications and it definitely should become more than a “new application wizard”. And because I’ve been playing Elder Scrolls. Fortunately, Michael convinced me not to name the project after the computer game. Thanks to Thom Mahoney for suggesting “App Scrolls” and saving me from a naming nightmare.

A new name and a new repository? I’m still in two minds as to if I should have done that. Oh well, its done now. App Scrolls is a cool name.

Why “Scrolls”?

Rails Wizard used the word “recipe” to describe each component or service that would be added to a new application. Talking with others at Engine Yard, the word “recipe” got confusing. There were wizard recipes and chef recipes. When you wanted to use Resque you needed both. “Scrolls!” seemed a fun alternative name.

Wizards use scrolls, not recipes. Every D&D person knows that.

Where are all the old scrolls?

For the initial release, I’ve moved many contributed “recipes” from the Rails Wizard into the scrolls/zzz archive folder. There are so many and I don’t yet know how to ensure they all work as expected. I intend to find a good way to allow rubygems to self-host their scroll.

Continuous Integration?

I mentioned that I like CI to be setup immediately for new applications. That’s next. Jenkins, Travis CI (when they release private application support), and what other CI systems should be supported?

I hope you use the App Scrolls for your next big or small Rails app and that its useful to you!

Validate and Save your Ruby in TextMate – with secret Rubinus superpowers

In some TextMate bundles, if you save a file it will also validate the file and show any syntax errors in a tooltip. This is awesome. (e.g. JavaScript and CoffeeScript)

So I added the same thing to my Ruby.tmbundle. Install this, save a dodgy Ruby file and you’ll now see something like:

Validate and Save - No Rubinius

Rubinius superpowers

Do you think the following syntax error tooltip is more useful?

Validate and Save - Rubinius installed

Yes it lovely, and the new Ruby.tmbundle will automatically do this if it can find rbx in your TextMate’s $PATH. Yeah yeah.

If you have Homebrew installed:

brew install rubinius

Then in TextMate, add your homebrew bin folder to the $PATH.

  • Go to TextMate’s Preferences (Cmd+,)
  • Go to “Advanced”, then “Shell Variables”
  • Edit the PATH variable, and add “:/path/to/homebrew/bin”

For example, if you have homebrew installed in ~/.homebrew then you might add :/Users/drnic/.homebrew/bin

. My complete $PATH in TextMate is:

/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:/usr/local/bin/:/Users/drnic/.homebrew/bin

Save a dodgy Ruby file and see the beautifully helpful syntax message.

Install Ruby.tmbundle

To install via Git:

mkdir -p ~/Library/Application\ Support/TextMate/Bundles
cd ~/Library/Application\ Support/TextMate/Bundles
git clone git://github.com/drnic/ruby-tmbundle.git "Ruby.tmbundle"
osascript -e 'tell app "TextMate" to reload bundles'

Using CoffeeScript in Rails and even on Heroku

I’m pretty excited about CoffeeScript as a clean-syntax replacement for pure JavaScript.

What is CoffeeScript?

Imagine all the syntactical delights of Ruby and Haml for your JavaScript. You write in a nice language, but get normal JavaScript at runtime. All whilst having full access to 3rd-party JavaScript libraries (jQuery, PrototypeJS), debugging support (it becomes pure, readable JavaScript), existing support from test suites (it’s normal JavaScript) and growing support from various text editors (TextMate, Vim, Emacs).

What simple delights?

No trailing semi-colons. No { some_code() } function/closure brackets. String interpolation. Multi-line strings. Explicit class syntax. Array slicing. An existential ? operator.

Scroll down the home page for awesome example after example.

These aren’t library extensions. This is clean, purposeful syntax.

You can play with the joyful syntax of CoffeeScript on the website. After reading the basic examples on the CoffeeScript home page, press “TRY COFFEESCRIPT” in the header menu.

As you play with the syntax, the equivalent JavaScript is printed on the right hand side (see image above).

How nice is that syntax? Very.

Installing CoffeeScript

  1. Install NodeJS
  2. Install CoffeeScript

For NodeJS (get latest release URL; using 0.1.31 as 0.1.32 doesn’t unpack for me):

cd /usr/local/src
wget http://nodejs.org/dist/node-v0.1.31.tar.gz
tar xfv node-v0.1.31.tar.gz
cd node-v0.1.31
./configure
make
sudo make install

For CoffeeScript (get latest release URL):

cd /usr/local/src
wget http://github.com/jashkenas/coffee-script/tarball/0.5.5
tar xfv jashkenas-coffee-script-bcf7b3f.tar.gz
cd jashkenas-coffee-script-bcf7b3f
sudo bin/cake install

Now test that everything is in place:

$ coffee --version
CoffeeScript version 0.5.5
$ coffee -e "sys: require 'sys'; sys.puts 'hello world\n'"
hello world

Phew!

Note, in the command-line/on the server, you are using the NodeJS JavaScript environment. It supports the CommonJS API for loading modules (normal JavaScript: var sys = require('sys')).

Um, but how do I use it in my web app?

Your application source code will have *.coffee files containing your sexy, short CoffeeScript. But at runtime, the browser needs the generated JavaScript.

I’ve been using the Jonas Nicklas’ bistro_car gem:

gem install bistro_car
mkdir -p app/scripts

In your Rails config/environment.rb file, add:

config.gem 'bistro_car'

And in your layouts, such as app/views/layouts/application.html.erb add to the <head> or the bottom:

<%= coffee_script_bundle %>

Now you’re good to go. Add your CoffeeScript files in app/scripts/*.coffee and they will be automatically available as JavaScript.

WARNING: Check your version of CoffeeScript

Check that this hasn’t happened:

$ coffee --version
CoffeeScript version 0.3.2
$ which coffee
/usr/bin/coffee

Arrgh, we should be using /usr/local/bin/coffee. bistro_car currently installs the old rubygem-based version of coffee-script; and you might be unlucky to have your $PATH find the wrong one first.

Either delete it (sudo rm /usr/bin/coffee and restart your shell) or make sure /usr/local/bin is earlier in your $PATH than /usr/bin, where RubyGems installed the old, unnecessary version of coffee command.

Let’s drink the CoffeeScript

Create a file app/scripts/application.coffee with contents:

powers: [1,2,3,4].map (i) -> i * i
alert powers

Load up a view in a browser and see [1,4,9,16]. You win! Throw in some jQuery/PrototypeJS/whatever. Beautiful.

View the source of the page, navigate to public/javascripts/bundle/default.js and you’ll see the generated source:

(function(){
  var powers;
  powers = [1, 2, 3, 4].map(function(i) {
    return i * i;
  });
  alert(powers);
})();

The problem: Heroku doesn’t have CoffeeScript installed

Heroku is a great place to host apps. Though it doesn’t have CoffeeScript installed so it cannot dynamically convert the *.coffee files into JavaScript.

If you want to use Heroku I guess we need to perform the conversion locally and deploy it.

But. In development and integration testing I want bistro_car’s dynamically generated default.js. In production, I need a cached version.

In application.html.haml I use (I can’t keep pretending I use erb):

- if Rails.env.production?
  = javascript_include_tag "coffeescripts"
- else
  = coffee_script_bundle

Now we’re just left with the hassle of automatically generating public/javascripts/coffeescripts.js.

First, a rake task. Second, a git pre-commit hook.

Create lib/tasks/bistro_car.rake:

desc "Generate the cached bundle/default.js file from app/scripts/*.coffee files"
task :bistro_car => :environment do
  path = "public/javascripts/coffeescripts.js"
  puts "Building *.coffee -> #{path}"
  File.open(path, "w") { |file| file << BistroCar::Bundle.new('default').to_javascript }
end

file "public/javascripts/coffeescripts.js" => Dir[File.join(Rails.root, 'app/scripts/*.coffee')] do |t|
  Rake::Task["bistro_car"].invoke
end

Now you can create coffeescripts.js and add it to the repo with:

rake public/javascripts/coffeescripts.js
git add public/javascripts/coffeescripts.js
git commit -m "Initial bundled coffeescripts file"

Now create .git/hooks/pre-commit:

#!/bin/sh

exec rake public/javascripts/coffeescripts.js

And make it executable (and git commit will invoke it automatically):

chmod +x .git/hooks/pre-commit

Phew.

Now, whenever you change a *.coffee script and you are about to commit it, the cached-production-only coffeescripts.js is automatically updated and included in the same commit.

Seems like a clean hack.

Summary

Why not make a library to do this? Well I’m hoping there is a better, cleaner way. Perhaps bistro_car can include a rails generator to package these bits and pieces itself, if my approach happens to be the best way.

Nonetheless, let history record that CoffeeScript is very cool though in the world of Heroku living with it is non-trivial at the moment.

Customized Google Forms

Google Form Customized

Google Forms are a great, free way to collect information from anyone, stored directly into a Google Spreadsheet, and then have Google notify you each time a form is submitted (optionally). The downside is that you can only use one of their pre-packaged themes. You can’t have your company logo and corporate “look”.

Pooey to Google, we say. So we wrote custom_google_forms.

Want to truly customized Google Forms? Fork this repository, customize the CSS, DOM and images, and deploy to Heroku. You can then host/customize any number of Google Forms.

For example, here is an original bare Google Form and here is a fully customized version, including form validation and thank you page.

It’s fully a dynamic, thin layer on top of Google Forms. If you change your Google Form, your custom form application automatically changes.

How to use Google Forms?

Perhaps read this docco? Essentially if you can access Google Spreadsheets, you can create a Google Form (which automatically creates and updates a Google Spreadsheet based on the fields and form submissions).

Fork and Customize

It’s really simple to play with and customize Google Forms. Clone the GitHub project and fire up the Rails server:

gem install github
cd ~/Sites
gh clone mocra/custom-google-forms yourdomain-google-forms
cd yourdomain-google-forms
script/server

Go to http://localhost:3000/google_forms/new and add a Google Form:

My Google Forms - New

The “Slug” is the URL path people will follow. That is, a slug “railsdev” maps to http://forms.mocra.com/railsdev.

The “Form Key” field takes either the Form URL or just the formkey parameter (e.g. dFo0LXQyYmlEV2dXcVJ6WjRweW9vRnc6MA) from when you view the live form.

Press “Create” and follow the slug url to view your Google Form through your own customized style.

Well, initially you’ll see the Mocra style. Let’s fix that.

Default Theme

First, fork the github repository so that you can push your style changes to your own repo.

gh fork
mate public/stylesheets/style.css

Now, edit public/stylesheets/style.css to your tastes.

Please don’t reuse our style. Aside from you looking a bit silly having the :mocra logo at the top, it would be weird if your forms looked like our forms. I guess you might as well have used the default Google Form themes?

Heroku Deployment

Let’s not get fancy, let’s just deploy your fork to Heroku. It’s free. It’s simple. You’ll be done in two minutes.

To get your copy of the application deployed to heroku:

gem install heroku
heroku create yourdomain-google-forms
git push heroku master
heroku rake db:migrate

Two optional environment variables:

heroku config:add GOOGLE_ANALYTICS='YOURCODE'

If specified, Google Analytics will be included on all pages (including the custom Google Forms page).

heroku config:add EXPECTED_DOMAIN='some.yourdomain.com'

If specified, ‘yourdomain-google-forms.heroku.com/someform’ automatically redirects to ‘some.domain.com/someform’.

For example, for http://forms.mocra.com we used:

heroku config:add GOOGLE_ANALYTICS='UA-5370510-4'
heroku config:add EXPECTED_DOMAIN='forms.mocra.com'

Getting Started

Once deployed, go to the /google_forms/new URL and add your first form. Once it is validated and added, you can start using it using the links shown.

Summary

Yay for free, customized Google Forms! Yay for free Heroku hosting!

Bonus: you can configure your Google Forms/Spreadsheet to notify you when new forms are submitted. That’s handy.

Thanks to Odin Dutton, our resident designer at Mocra, who did a great job applying our new theme to the Google Forms DOM structure. It’s wonderful having him on the team!

Hacking someone’s gem with github and gemcutter

gemcutter

Ever used a rubygem, found a bug, and just wanted to quickly bust out the big guns and fix it quickly?

The gem command doesn’t come packed with a way to find the original source repository for a gem. At best, most gems at least come bundled with the complete source, tests and documentation. Some gems don’t. Fair enough, since having access to the complete source via the gem still doesn’t allow you to fix a bug and share it with the world.

For that you access to the repo, a quick way to fork it, and a post-github way to share a gem version from yours truly.

The github gem and gemcutter are the modern day tools of master hackermanship.

Instant forking fun

Let’s say you find a bug in a gem, say rails, and you want to go to town on its source.

You know the gem is called rails but you’ve no idea what the github repo is called. Never fear.

$ gem sources -a http://gemcutter.org
$ sudo gem install github
$ gh clone --search rails
Select a repository to clone:
1.  rails/rails                         # Ruby on Rails
2.  technoweenie/restful-authentication # Generates common user ...
3.  justinfrench/formtastic             # A Rails form builder plugin ...
?

Press 1 and you’ll get a clone of rails/rails.

Alternately, if you want a fork or you know the exact user/repo already:

$ gh clone rails/rails

Now, fork your own version:

$ cd rails
$ gh fork

You now have your own fork. The origin remote also now points to your fork rather than the rails/rails repository:

$ git remote show origin
* remote origin
  Fetch URL: git@github.com:drnic/rails.git
  Push  URL: git@github.com:drnic/rails.git

So, make your changes, push them. Send a pull request or github issue or lighthouse ticket or what have you.

Want to get to the github project home page for your fork?

$ gh home

Instant gem sharing

Let’s say you patched the rails gem itself but you want to share your changes via your own gem.

In the olden days, github did this for you. Now you use gemcutter, and a little manual effort to do your own renaming.

First, install the gems locally, use them, and make sure all is good.

For rails, you install the edge gems (3.0.pre) with:

$ rake install

You can’t see ‘rake install’ in the rake -T list (hence my patch), but I think the following expression displays all tasks regardless if they have a description or not:

$ rake -P | grep "^r"

Rails is composed of several gems, unlike most projects that are distributed as a single gem. Here we want to share our commit within a new drnic-rails gem, but not touch the others.

Edit the railties/rails.gemspec file from:

Gem::Specification.new do |s|
  s.platform = Gem::Platform::RUBY
  s.name = 'rails'
  s.version = '3.0.pre'
...

and give your personal gem a new name:

Gem::Specification.new do |s|
  s.platform = Gem::Platform::RUBY
  s.name = 'drnic-rails'
  s.version = '3.0.pre'

To build and distribute the new gem:

$ gem build railties/rails.gemspec
$ sudo gem install gemcutter
$ gem push drnic-rails-3.0.pre.gem
  Pushing gem to Gemcutter...
  Successfully registered gem: drnic-rails (3.0.pre)

Follow any first-time gemcutter instructions and SUCCESS! Now I have my own drnic-rails gem.

Summary

To find, clone, and fork any rubygem that is hosted on github:

$ sudo gem install drnic-github
$ gh clone --search rails
$ gh fork

To personalise the gem and share it on gemcutter:

> edit the project.gemspec to have a unique name, e.g. yourname-project
$ gem build project.gemspec
$ sudo gem install gemcutter
$ gem push yourname-project-1.0.0.gem

I think this makes it much easier, faster and more fun to hack other people’s stuff.