Dr Nic

Instant new Rails applications with the App Scrolls

When I start a new project I want to start how I plan to finish.

If I intend to write integration tests then I want to start them immediately. If I intend to set up continuous integration then I want CI setup immediately. If I intend to deploy the app somewhere then I want to deploy it immediately. If I intend to use some background workers, I want to know that they are setup immediately.

I want it all setup as easily as I can generate a new Rails app. They shouldn’t be extra work later. I’ll be busy then.

Assuming you want the same simplicity and speed, then let me share with you the App Scrolls!

gem install appscrolls
scrolls new mydemoapp -s twitter_bootstrap unicorn postgresql resque github eycloud

This one command will:

  • create a new Rails application called “mydemoapp”
  • install Twitter Bootstrap (you choose fixed or floating) and some other Rails basic tweaks
  • sets up Unicorn and Postgresql in the application
  • sets up Resque and Redis for background workers
  • hosts the project on GitHub (public or private repo)
  • deploys the application on Engine Yard Cloud

These are the magical App Scrolls!

Can you run the command above, or do you need a video to see it happen? Let me know.

Scrolls are interactive

$ scrolls new myapp -s twitter_bootstrap github resque postgresql unicorn eycloud
...
   question  Create a GitHub repository?
          1)  Public
          2)  Private
      github  Enter your selection: 1
...
      resque  Enter a secret string for the route /resque/YOUR-SECRET-STRING: mysecret
...
    question  Which Twitter Bootstrap layout?
          1)  Fluid
          2)  Fixed
  twitter_bootstrap  Enter your selection: 2
...
    question  Select application cluster configuration?
          1)  Basic - 1 app VM (5 CPU-based processes) & DB Master VM
          2)  Pro   - 3 app highly-available VMs (15 CPU-based processes) & DB Master VM
          3)  Solo  - 1 VM for app processes, DB and other services
     eycloud  Enter your selection: 1

Deploy automatically

I perfer to host on Engine Yard Cloud and I work there. So I added support to Engine Yard Cloud to all the scrolls and it will automatically boot a new environment and deploy your new application. Thanks to David Calavera’s unofficial ey_cli tool and the unofficial github-ruby CLI for making this possible!

Which Scrolls are supported?

Lots of magical scrolls are available!

administration: active_admin
assets: jquery, prototype
deployment: eycloud, eycloud_recipes_on_deploy, git, github, passenger, unicorn
persistence: mysql, postgresql, redis, sqlite3
stylesheet: twitter_bootstrap
templating: simple_form
testing: capybara, cucumber, rspec, test_unit
worker: delayed_job, resque
other: env_yaml, guard, rails_basics, split

I might blog about individual combinations later. My favourite is guard, which magically combines with other scrolls you choose.

Heroku or CloudFoundry?

If you’re a fan/employee of Salesforce/Heroku or VMWare/CloudFoundry or other platform, can I please ask for your help to make the current set of scrolls work? I’ll pair with you to get you started if you need it; though I think the code base and the scrolls are relatively self explanatory.

Isn’t this like Rails Wizard?

Yes! Before the App Scrolls, there was Michael Bleigh’s Rails Wizard. He created the first version during the 2010 Rails Rumble [Intridea blog post], and iterated on it a few times. I especially like that he extracted the project into two codebases – the CLI/scrolls and the web version.

I renamed it because ultimately I think it could support more than Rails applications and it definitely should become more than a “new application wizard”. And because I’ve been playing Elder Scrolls. Fortunately, Michael convinced me not to name the project after the computer game. Thanks to Thom Mahoney for suggesting “App Scrolls” and saving me from a naming nightmare.

A new name and a new repository? I’m still in two minds as to if I should have done that. Oh well, its done now. App Scrolls is a cool name.

Why “Scrolls”?

Rails Wizard used the word “recipe” to describe each component or service that would be added to a new application. Talking with others at Engine Yard, the word “recipe” got confusing. There were wizard recipes and chef recipes. When you wanted to use Resque you needed both. “Scrolls!” seemed a fun alternative name.

Wizards use scrolls, not recipes. Every D&D person knows that.

Where are all the old scrolls?

For the initial release, I’ve moved many contributed “recipes” from the Rails Wizard into the scrolls/zzz archive folder. There are so many and I don’t yet know how to ensure they all work as expected. I intend to find a good way to allow rubygems to self-host their scroll.

Continuous Integration?

I mentioned that I like CI to be setup immediately for new applications. That’s next. Jenkins, Travis CI (when they release private application support), and what other CI systems should be supported?

I hope you use the App Scrolls for your next big or small Rails app and that its useful to you!

Using CoffeeScript in Rails and even on Heroku

I’m pretty excited about CoffeeScript as a clean-syntax replacement for pure JavaScript.

What is CoffeeScript?

Imagine all the syntactical delights of Ruby and Haml for your JavaScript. You write in a nice language, but get normal JavaScript at runtime. All whilst having full access to 3rd-party JavaScript libraries (jQuery, PrototypeJS), debugging support (it becomes pure, readable JavaScript), existing support from test suites (it’s normal JavaScript) and growing support from various text editors (TextMate, Vim, Emacs).

What simple delights?

No trailing semi-colons. No { some_code() } function/closure brackets. String interpolation. Multi-line strings. Explicit class syntax. Array slicing. An existential ? operator.

Scroll down the home page for awesome example after example.

These aren’t library extensions. This is clean, purposeful syntax.

You can play with the joyful syntax of CoffeeScript on the website. After reading the basic examples on the CoffeeScript home page, press “TRY COFFEESCRIPT” in the header menu.

As you play with the syntax, the equivalent JavaScript is printed on the right hand side (see image above).

How nice is that syntax? Very.

Installing CoffeeScript

  1. Install NodeJS
  2. Install CoffeeScript

For NodeJS (get latest release URL; using 0.1.31 as 0.1.32 doesn’t unpack for me):

cd /usr/local/src
wget http://nodejs.org/dist/node-v0.1.31.tar.gz
tar xfv node-v0.1.31.tar.gz
cd node-v0.1.31
./configure
make
sudo make install

For CoffeeScript (get latest release URL):

cd /usr/local/src
wget http://github.com/jashkenas/coffee-script/tarball/0.5.5
tar xfv jashkenas-coffee-script-bcf7b3f.tar.gz
cd jashkenas-coffee-script-bcf7b3f
sudo bin/cake install

Now test that everything is in place:

$ coffee --version
CoffeeScript version 0.5.5
$ coffee -e "sys: require 'sys'; sys.puts 'hello world\n'"
hello world

Phew!

Note, in the command-line/on the server, you are using the NodeJS JavaScript environment. It supports the CommonJS API for loading modules (normal JavaScript: var sys = require('sys')).

Um, but how do I use it in my web app?

Your application source code will have *.coffee files containing your sexy, short CoffeeScript. But at runtime, the browser needs the generated JavaScript.

I’ve been using the Jonas Nicklas’ bistro_car gem:

gem install bistro_car
mkdir -p app/scripts

In your Rails config/environment.rb file, add:

config.gem 'bistro_car'

And in your layouts, such as app/views/layouts/application.html.erb add to the <head> or the bottom:

<%= coffee_script_bundle %>

Now you’re good to go. Add your CoffeeScript files in app/scripts/*.coffee and they will be automatically available as JavaScript.

WARNING: Check your version of CoffeeScript

Check that this hasn’t happened:

$ coffee --version
CoffeeScript version 0.3.2
$ which coffee
/usr/bin/coffee

Arrgh, we should be using /usr/local/bin/coffee. bistro_car currently installs the old rubygem-based version of coffee-script; and you might be unlucky to have your $PATH find the wrong one first.

Either delete it (sudo rm /usr/bin/coffee and restart your shell) or make sure /usr/local/bin is earlier in your $PATH than /usr/bin, where RubyGems installed the old, unnecessary version of coffee command.

Let’s drink the CoffeeScript

Create a file app/scripts/application.coffee with contents:

powers: [1,2,3,4].map (i) -> i * i
alert powers

Load up a view in a browser and see [1,4,9,16]. You win! Throw in some jQuery/PrototypeJS/whatever. Beautiful.

View the source of the page, navigate to public/javascripts/bundle/default.js and you’ll see the generated source:

(function(){
  var powers;
  powers = [1, 2, 3, 4].map(function(i) {
    return i * i;
  });
  alert(powers);
})();

The problem: Heroku doesn’t have CoffeeScript installed

Heroku is a great place to host apps. Though it doesn’t have CoffeeScript installed so it cannot dynamically convert the *.coffee files into JavaScript.

If you want to use Heroku I guess we need to perform the conversion locally and deploy it.

But. In development and integration testing I want bistro_car’s dynamically generated default.js. In production, I need a cached version.

In application.html.haml I use (I can’t keep pretending I use erb):

- if Rails.env.production?
  = javascript_include_tag "coffeescripts"
- else
  = coffee_script_bundle

Now we’re just left with the hassle of automatically generating public/javascripts/coffeescripts.js.

First, a rake task. Second, a git pre-commit hook.

Create lib/tasks/bistro_car.rake:

desc "Generate the cached bundle/default.js file from app/scripts/*.coffee files"
task :bistro_car => :environment do
  path = "public/javascripts/coffeescripts.js"
  puts "Building *.coffee -> #{path}"
  File.open(path, "w") { |file| file << BistroCar::Bundle.new('default').to_javascript }
end

file "public/javascripts/coffeescripts.js" => Dir[File.join(Rails.root, 'app/scripts/*.coffee')] do |t|
  Rake::Task["bistro_car"].invoke
end

Now you can create coffeescripts.js and add it to the repo with:

rake public/javascripts/coffeescripts.js
git add public/javascripts/coffeescripts.js
git commit -m "Initial bundled coffeescripts file"

Now create .git/hooks/pre-commit:

#!/bin/sh

exec rake public/javascripts/coffeescripts.js

And make it executable (and git commit will invoke it automatically):

chmod +x .git/hooks/pre-commit

Phew.

Now, whenever you change a *.coffee script and you are about to commit it, the cached-production-only coffeescripts.js is automatically updated and included in the same commit.

Seems like a clean hack.

Summary

Why not make a library to do this? Well I’m hoping there is a better, cleaner way. Perhaps bistro_car can include a rails generator to package these bits and pieces itself, if my approach happens to be the best way.

Nonetheless, let history record that CoffeeScript is very cool though in the world of Heroku living with it is non-trivial at the moment.

Customized Google Forms

Google Form Customized

Google Forms are a great, free way to collect information from anyone, stored directly into a Google Spreadsheet, and then have Google notify you each time a form is submitted (optionally). The downside is that you can only use one of their pre-packaged themes. You can’t have your company logo and corporate “look”.

Pooey to Google, we say. So we wrote custom_google_forms.

Want to truly customized Google Forms? Fork this repository, customize the CSS, DOM and images, and deploy to Heroku. You can then host/customize any number of Google Forms.

For example, here is an original bare Google Form and here is a fully customized version, including form validation and thank you page.

It’s fully a dynamic, thin layer on top of Google Forms. If you change your Google Form, your custom form application automatically changes.

How to use Google Forms?

Perhaps read this docco? Essentially if you can access Google Spreadsheets, you can create a Google Form (which automatically creates and updates a Google Spreadsheet based on the fields and form submissions).

Fork and Customize

It’s really simple to play with and customize Google Forms. Clone the GitHub project and fire up the Rails server:

gem install github
cd ~/Sites
gh clone mocra/custom-google-forms yourdomain-google-forms
cd yourdomain-google-forms
script/server

Go to http://localhost:3000/google_forms/new and add a Google Form:

My Google Forms - New

The “Slug” is the URL path people will follow. That is, a slug “railsdev” maps to http://forms.mocra.com/railsdev.

The “Form Key” field takes either the Form URL or just the formkey parameter (e.g. dFo0LXQyYmlEV2dXcVJ6WjRweW9vRnc6MA) from when you view the live form.

Press “Create” and follow the slug url to view your Google Form through your own customized style.

Well, initially you’ll see the Mocra style. Let’s fix that.

Default Theme

First, fork the github repository so that you can push your style changes to your own repo.

gh fork
mate public/stylesheets/style.css

Now, edit public/stylesheets/style.css to your tastes.

Please don’t reuse our style. Aside from you looking a bit silly having the :mocra logo at the top, it would be weird if your forms looked like our forms. I guess you might as well have used the default Google Form themes?

Heroku Deployment

Let’s not get fancy, let’s just deploy your fork to Heroku. It’s free. It’s simple. You’ll be done in two minutes.

To get your copy of the application deployed to heroku:

gem install heroku
heroku create yourdomain-google-forms
git push heroku master
heroku rake db:migrate

Two optional environment variables:

heroku config:add GOOGLE_ANALYTICS='YOURCODE'

If specified, Google Analytics will be included on all pages (including the custom Google Forms page).

heroku config:add EXPECTED_DOMAIN='some.yourdomain.com'

If specified, ‘yourdomain-google-forms.heroku.com/someform’ automatically redirects to ‘some.domain.com/someform’.

For example, for http://forms.mocra.com we used:

heroku config:add GOOGLE_ANALYTICS='UA-5370510-4'
heroku config:add EXPECTED_DOMAIN='forms.mocra.com'

Getting Started

Once deployed, go to the /google_forms/new URL and add your first form. Once it is validated and added, you can start using it using the links shown.

Summary

Yay for free, customized Google Forms! Yay for free Heroku hosting!

Bonus: you can configure your Google Forms/Spreadsheet to notify you when new forms are submitted. That’s handy.

Thanks to Odin Dutton, our resident designer at Mocra, who did a great job applying our new theme to the Google Forms DOM structure. It’s wonderful having him on the team!

First look at rails 3.0.pre

This article is out of date in some aspects. See comments, and perhaps this summary of reading materials about Rails 3.

3.pre

Today I had my first look at rails 3.0.pre and below are the sequence of steps I had to take to create a rails 3.0.pre application, and get it’s generators to work.

Why was I looking at the top-secret, yet open-source Rails 3.0? Their generators are being migrated over to Thor and I wanted to see them in action. I was thinking I might migrate newgem to use Thor too.

Here’s a quick poke around of getting started and interesting things I found. Any hiccups and workarounds are meant as a guide to pre-pre-3.0 users/developers and not as a criticism of rails-core. Rails 3.0 is looking shiny and awesome.

NOTE: Since this is a “how to install and use” rails 3.0 edge, which is still in heavy development, this set of instructions might break. Let’s hope not.

Getting Started

As of today, you cannot install 3.0.pre from rubygems [1]. So, let’s install them from source. Which is handy, you might like to patch something.

$ cd ~/gems
$ git clone git://github.com/rails/rails.git
use_ruby_191 *

[*] If you are on OS X Snow Leopard I think you can ignore this. Otherwise, since you don’t have the 3.0.pre gems installed, you’re about to hit bump #1. Ruby 1.8.6 doesn’t have Symbol#to_proc but it’s required to create a rails app. This means you’ll need to be able to switch to another version of ruby temporarily if you’re on ruby 1.8.6 [2].

cd ~/Sites
ruby ~/gems/rails/railties/bin/rails

Oooh, look at all the new options! Some new ones are:

-F, [--freeze]             # Freeze Rails in vendor/rails from the gems
-O, [--skip-activerecord]  # Skip ActiveRecord files
-T, [--skip-testunit]      # Skip TestUnit files
-J, [--skip-prototype]     # Skip Prototype files

The -D, --with-dispatchers flags have been removed. --freeze isn’t new, but -F is.

So, to create an app, I dutifully used:

ruby ~/gems/rails/railties/bin/rails edgerailsapp -F

BAM! Fail. The -F option to freeze/vendor rails fails without the gems installed. So don’t use it.

ruby ~/gems/rails/railties/bin/rails edgerailsapp
ln -s ~/gems/rails vendor/rails

If you’re on Windows without the symlink command ln, then copy the downloaded rails source into vendor/rails.

Fetch Rails’ dependencies

Rails 3.0 source uses the new bundler project to describe its own dependencies. From Nick Quaranto’s article on bundler, get the latest:

cd ~/gems
git clone git://github.com/wycats/bundler
cd bundler
sudo rake install

Now, back in your app, you need to install some rails dependencies here too. It’s a good chance to see how you’ll bundle gem dependencies in the future.

$ cd ~/Sites/edgerailsapp

Change the Gemfile in your project to the following:

gem "rack",          "1.0.1"
gem "rack-mount",    :git => "git://github.com/rails/rack-mount.git"
gem "rack-test",     "~> 0.5.0"
gem "erubis",        "~> 2.6.0"
gem "arel",          :git => "git://github.com/rails/arel.git"
gem "sqlite3-ruby"
gem "rails", "3.0.pre", :git => "git://github.com/rails/rails.git"

Welcome to the future of gem dependencies for rails apps. Ultimately you won’t need to manually add these lines yourself. When rails is distributed as gems it will automatically install these for you, I assume/hope/guess. But for today, you seem to need them.

Now locally (within your app) install these gems:

$ gem bundle

If you get “can’t convert Pathname into String” then revert to ruby 1.8.X and reinstall bundler into your 1.8 gem cache.

Phew. Ooh, my god. Phew. Only now will script/generate work.

$ script/generate

For me, this outputs:

Please select a generator.
Builtin: app, controller, generator, helper, integration_test, mailer, metal, migration, model, model_subclass, observer, performance_test, plugin, resource, scaffold, scaffold_controller, session_migration, stylesheets.
Others: app_layout:app_layout, check_migration_version:check_migration_version, home_route:home_route.

The “Builtin” generators are the latest and greatest in Thor technology. Rails 3.0 no longer uses its own generator but is built upon Thor.

For example, our old favourite model generator works thusly:

$ script/generate model Post title:string --no-fixture
    invoke  active_record
    create    db/migrate/20091103030824_create_posts.rb
    create    app/models/post.rb
    invoke    test_unit
    create      test/unit/post_test.rb

Interestingly, --no-fixture isn’t mentioned in the usage information for script/generate model. It mentions the --fixture flag, but I had to guess that --no-fixture was also supported.

Hmm, I want to use rspec. So, let’s destroy these files:

$ script/destroy model Post title:string
      invoke  active_record
.../vendor/gems/dirs/rails/railties/lib/rails/generators/active_record.rb:14:in
  `next_migration_number': uninitialized constant ActiveRecord::Base (NameError)

Oh well.

What if I wanted to run rspec and cucumber generators, for example, against an edge rails app?

Rails 2 generators

The “Others” generators are my own local generators from ~/.rails/generators. Amusingly, instead of app_layout it is called app_layout:app_layout. Not surprisingly at all, if I try to run the rails 2 generator it fails:

$ script/generate app_layout:app_layout
[WARNING] Could not load generator at "/Users/drnic/.rails/generators/app_layout/app_layout_generator.rb". Error: uninitialized constant Rails::Generator
Could not find generator app_layout:app_layout.

Poop.

Note, I have rspec, rspec-rails and cucumber gems installed locally but I cannot see their rails generators above. Rails 3 doesn’t look for generators in the same way and old Rails 2 generators don’t work anymore.

That’s the news: every rails 2 generator is broken.

When I start to migrate some of mine I’ll post about it. In the meantime, José Valim has written some introduction thoughts on using Thor as a generator.

You can also probably learn about how to write rails 3.0 generators by looking at the source code for the new generators like rails, model (see main and test_unit), and scaffold/resource.

Finally, José Valim has a sample Rails 3 app with some vendored generators in it.

These are the things I’m researching now.

Summary

This article is long, mostly because rails 3.0.pre hasn’t been released as a set of RubyGems. If it had, then all the dependencies would be installed automatically.

It also introduces gem/plugin writers to the first upgrade issue: your current generators are neither discovered nor work by a rails 3.0 app. We’re all clever cookies, so here’s hoping we can figure out the upgrade path and that it’s simple enough to not be the topic of Dan Brown’s next book.

Footnotes

[1] Two portions of rails 3.0.pre are available as pre-release gems: activesupport (which is now very modularised and only loads up the parts that you require) and activemodel (which is shiny and new and hence completely safe for rails-core to release).

[2] There are two popular ways to have easy, non-intrusive access to alternate versions of ruby: rvm and ruby_switcher.sh.

Rails themes can remember things

I was getting annoyed at having to remember all the csspath/xpath expressions for a theme I reused in a new project.

So, install_theme now helps each theme folder remember the previously used settings.

gem install install_theme  # version 0.7.0+
install_theme path/to/rails_app path/to/template #content_box \
      --partial "header:#header h2" \
      --partial sidebar:#sidebar"

Now, the next time you apply that same theme to another project you don’t need to mention “#content_box” or use the --partial flags:

install_theme path/to/another_rails_app path/to/template

How?

If you squint your eyes just right, you’ll notice that your original template folder now has an install_theme.yml file. It contains your original settings. You can imagine for yourself how the rest of the “themes can remember things” magic might work.

Templates “For Ruby on Rails”

If you are a template maker, you can now easily make your HTML template “For Ruby on Rails” by including an install_theme.yml file. Think of the children.

Major change

I changed the order of the first two arguments. In future, the path/to/template will be optional. Why? Imagine if each theme you ever used was cached in ~/.install_theme/themes and you could select a theme from a list or by --theme theme_name. That seems neat.