Dr Nic

Hacking someone’s gem with github and gemcutter

gemcutter

Ever used a rubygem, found a bug, and just wanted to quickly bust out the big guns and fix it quickly?

The gem command doesn’t come packed with a way to find the original source repository for a gem. At best, most gems at least come bundled with the complete source, tests and documentation. Some gems don’t. Fair enough, since having access to the complete source via the gem still doesn’t allow you to fix a bug and share it with the world.

For that you access to the repo, a quick way to fork it, and a post-github way to share a gem version from yours truly.

The github gem and gemcutter are the modern day tools of master hackermanship.

Instant forking fun

Let’s say you find a bug in a gem, say rails, and you want to go to town on its source.

You know the gem is called rails but you’ve no idea what the github repo is called. Never fear.

$ gem sources -a http://gemcutter.org
$ sudo gem install github
$ gh clone --search rails
Select a repository to clone:
1.  rails/rails                         # Ruby on Rails
2.  technoweenie/restful-authentication # Generates common user ...
3.  justinfrench/formtastic             # A Rails form builder plugin ...
?

Press 1 and you’ll get a clone of rails/rails.

Alternately, if you want a fork or you know the exact user/repo already:

$ gh clone rails/rails

Now, fork your own version:

$ cd rails
$ gh fork

You now have your own fork. The origin remote also now points to your fork rather than the rails/rails repository:

$ git remote show origin
* remote origin
  Fetch URL: git@github.com:drnic/rails.git
  Push  URL: git@github.com:drnic/rails.git

So, make your changes, push them. Send a pull request or github issue or lighthouse ticket or what have you.

Want to get to the github project home page for your fork?

$ gh home

Instant gem sharing

Let’s say you patched the rails gem itself but you want to share your changes via your own gem.

In the olden days, github did this for you. Now you use gemcutter, and a little manual effort to do your own renaming.

First, install the gems locally, use them, and make sure all is good.

For rails, you install the edge gems (3.0.pre) with:

$ rake install

You can’t see ‘rake install’ in the rake -T list (hence my patch), but I think the following expression displays all tasks regardless if they have a description or not:

$ rake -P | grep "^r"

Rails is composed of several gems, unlike most projects that are distributed as a single gem. Here we want to share our commit within a new drnic-rails gem, but not touch the others.

Edit the railties/rails.gemspec file from:

Gem::Specification.new do |s|
  s.platform = Gem::Platform::RUBY
  s.name = 'rails'
  s.version = '3.0.pre'
...

and give your personal gem a new name:

Gem::Specification.new do |s|
  s.platform = Gem::Platform::RUBY
  s.name = 'drnic-rails'
  s.version = '3.0.pre'

To build and distribute the new gem:

$ gem build railties/rails.gemspec
$ sudo gem install gemcutter
$ gem push drnic-rails-3.0.pre.gem
  Pushing gem to Gemcutter...
  Successfully registered gem: drnic-rails (3.0.pre)

Follow any first-time gemcutter instructions and SUCCESS! Now I have my own drnic-rails gem.

Summary

To find, clone, and fork any rubygem that is hosted on github:

$ sudo gem install drnic-github
$ gh clone --search rails
$ gh fork

To personalise the gem and share it on gemcutter:

> edit the project.gemspec to have a unique name, e.g. yourname-project
$ gem build project.gemspec
$ sudo gem install gemcutter
$ gem push yourname-project-1.0.0.gem

I think this makes it much easier, faster and more fun to hack other people’s stuff.

Rails themes can remember things

I was getting annoyed at having to remember all the csspath/xpath expressions for a theme I reused in a new project.

So, install_theme now helps each theme folder remember the previously used settings.

gem install install_theme  # version 0.7.0+
install_theme path/to/rails_app path/to/template #content_box \
      --partial "header:#header h2" \
      --partial sidebar:#sidebar"

Now, the next time you apply that same theme to another project you don’t need to mention “#content_box” or use the --partial flags:

install_theme path/to/another_rails_app path/to/template

How?

If you squint your eyes just right, you’ll notice that your original template folder now has an install_theme.yml file. It contains your original settings. You can imagine for yourself how the rest of the “themes can remember things” magic might work.

Templates “For Ruby on Rails”

If you are a template maker, you can now easily make your HTML template “For Ruby on Rails” by including an install_theme.yml file. Think of the children.

Major change

I changed the order of the first two arguments. In future, the path/to/template will be optional. Why? Imagine if each theme you ever used was cached in ~/.install_theme/themes and you could select a theme from a list or by --theme theme_name. That seems neat.

Install any HTML theme/template into your Rails app

theme applied and menu update

Have you ever even bothered to Google for “rails html template”? There are millions of “WordPress themes” you can download for free or less than $100, a thousand times more static HTML templates, but never any category of template called “Ruby on Rails theme”. 24 millions results for Googling single column html theme.

So we’re only left with HTML templates. Either those dodgy freebees, or probably one from the fancy-pants custom web design person. But how do we install them to our Rails apps?

I don’t know. It sucks. And it takes more time than it should. Here’s my idea – a tool to install any HTML template into your Rails app. To treat any HTML template as if it was a “Ruby on Rails HTML Template”.

So I’ve started to try and make any “HTML Template” into a “Ruby on Rails Template” with the helper app install_theme.

What’s it do?

Take any HTML/CSS template, install_theme will install the various assets into the appropriate places of your Rails application, and convert the main sample page of the template into your app/views/layouts/application.html.erb (or .haml). Easy peasy.

Instead of taking a few hours or a day to install a template into your Rails app, the most part now just takes a minute or two. Into either ERB or Haml. Repeatable if the original HTML/CSS template changes.

Consider a free admin template Refreshed [download].

refreshed theme

Installing a theme for fun and profit into a fresh rails app:

$ gem install install_theme
$ rails my_app
$ cd my_app
$ install_theme . path/to/theme/folder ".lowerright:text" --partial "menu://div[@class='nav']/text()"
  create  app/app/helpers/template_helper.rb
  create  app/controllers/original_template_controller.rb
  create  app/helpers/template_helper.rb
  create  app/views/layouts/_menu.html.erb
  create  app/views/layouts/application.html.erb
  create  app/views/original_template/index.html.erb
  create  public/images/footer.png
  ...
  create  public/stylesheets/style.css

Your theme has been installed into your app.

When you launch the app, it will be instantly themed. The section of the original template with DOM path .lowerright will be removed and replaced by your rendered actions.

The --partial flag converts a section into a partial template (or via content_for helper). More on this in a minute.

Note: the example above uses both CSS path and XPath expressions. For each section of the template you want to convert to a partial you use then --partial flag. The argument is “label:xpath” or “label:csspath”. So either --partial "header://div[@id='header']/h2" or --partial "header:#header h2".

Here are the content and partial selections using CSSpath:

$ install_theme . path/to/theme/folder ".lowerright:text" --partial "menu:.nav:text"

refreshed theme - identifying partials

Here are the content and partial selections using XPath:

$ install_theme . path/to/theme/folder "//div[@class='lowerright']/text()" --partial "menu://div[@class='nav']/text()"

refreshed theme - identifying partials

Overriding the theme partials

Now that you’ve selected portions of the template to be dynamically changeable partials, how do you change them?

  1. Use <% content_for :menu do %> ... <% end %> from any view template
  2. Create a _menu.html.erb partial in your controller’s views folder, e.g. app/views/posts/_menu.html.erb
  3. Modify the _menu.html.erb partial in the app/views/layouts folder. This is the default source.

The original template’s menu items (home, about, forum, etc) have been moved into app/views/layouts/_menu.html.erb. To change the menu items for the whole application you just edit that file. For this template, it will look like:

<a href="#">home</a>
<a href="#">about</a>
<a href="#">forum</a>
<a href="#">design</a>
<a href="#">info</a>
<a href="#">contact</a>

This is the extracted content of the .nav DOM element. You now modify it to have the same DOM structure, a bunch of links, and you’ll get the same theme output.

Let’s change the menu across the entire application. Edit app/views/layouts/_menu.html.erb:

<%= link_to "home", "/" %>
<%= link_to "posts", posts_path %>
<%= link_to "new post", new_post_path %>

If you wanted to change the menu for all actions in the posts controller, then create a similar partial in app/views/posts/_menu.html.erb.

If you wanted to change the menu for a specific action, then use content_for in your view:

<% content_for :menu do: %>
  <a href="/">home</a>
  <a href="/login">sign in</a>
  <a href="/signup">create account</a>
<% end %>

Haml

I use Haml and I like it. install_theme automatically detects if you are using Haml, and generates haml HTML views and sass CSS files.

$ gem install drnic-haml --source http://gemcutter.org  # see below
$ rails my_haml_app
$ cd my_haml_app
$ haml --rails .
$ install_theme . path/to/theme/folder ".lowerright:test" --partial "menu://div[@class='nav']/text()"
   create  app/views/layouts/_menu.html.haml
   create  app/views/layouts/application.html.haml
   create  app/views/original_template/index.html.haml
   create  public/stylesheets/sass/style.sass

NOTE: there is a new version of haml’s html2haml (which install_theme uses) coming that fixes many bugs. In the short term, use the drnic-haml above.

Where’d my original content go?

Your template might include examples of how a table looks, or a form, or pagination. It would good if they weren’t lost on the chopping floor.

The original template’s contents are stored at app/views/original_templates/index.html.erb and viewable at http://localhost:3000/original_template

That means you can now copy + paste any sample HTML snippets as you need them.

How it works?

Look inside the generated application.html.erb file and you’ll see the following for each named partial:

<%= yield(:menu) || render_or_default('menu') %>

The yield(:menu) enables the content_for helper to override the partials.

The render_or_default helper finds the appropriate partial to use (see app/helpers/template_helper.rb for source).

The Future

Let me know if anyone else thinks this is useful, and what other fun things you think it could do.

Migrating project websites to github pages with sake tasks, new websites with jekyll_generator

Its almost Christmas time and that means presents.

It also means that sometime between today and December 25th you need to go out and buy other people some presents because you haven’t done it yet.

But there’s someone else special in your life that deserves an Xmas gift this year. That special someone is your open source projects.

You might think, “But, Dr Nic, what do I get for the open source project that already has everything? What gift would my open source projects possible appreciate?”

Its true – this year has been a boon for open source. You probably migrated your projects from SVN to Git this year. Specifically, you probably clicked the “create a new one” link on your GitHub home page a lot this year: either to migrate your old projects to github or start new ones. Lots of new ones.

And now that your code is on github, your README file is rendered beautifully on the home page, so you were more inclined to rename it to README.markdown or README.textile and thusforthly fill it full of education information about your projects.

More people would then instantaneously figure out what the f@#$ your project actually does, thus more people used it, thus more people wrote patches or forked your code and sent Pull Requests.

All round, in 2008, I bet your open source projects had a sweet year.

Nonetheless, it is Christmas time and you should now get them a present. Something they wouldn’t get for themselves. Something you wouldn’t have done for them except for the free loving spirit of Christmas.

The Gift for all Open Source projects

A website.

Oh sure, github renders your README.markdown file. Yeah, yeah, github gives you a wiki for your project. Sure, sure, Google Groups let you communicate with your co-developers and users. Certainly, you don’t need a website for your project.

That’s why its a Christmas gift. You’re going to do it because you care.

Websites sell your project. READMEs and Wikis educate. The project website will sell it and make people want to use your stuff.

A website could have a blog with an RSS/atom feed. Then you could post updates about your project. People could subscribe and also leave comments. Oh the novelty.

And if it only took 5 minutes to get all this setup – the website code, the blog engine, the RSS feed, the comments, and published to its own hosted server – then that would just be oh so sweet.

So we’re going to do some craft for our Xmas present. A little DIY project, if you will. You’ll need a few things that you’ll find around your house, a command-line interface, and a beer or perhaps some port or sherry. Christmas cake is good too.

(more…)

Future proofing your Ruby code. Ruby 1.9.1 is coming.

Bugger.

I’m a Ruby monogamist. I use the Ruby that comes with Leopard (ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0]). Oh sure I’ve cheated on my Ruby a couple of times. It was just sex, I didn’t fall in love, I promise.

My machine has had various versions of jRuby, Rubinius and MacRuby installed at various times, but I don’t think it’s ever had two working copies of Ruby MRI (Matz Ruby Implementation) at a time.

Ok, that’s a lie. On Christmas Day 2007, Matz released Ruby 1.9. Yes I was a deviant. I installed it. But I didn’t inhale. ruby19 sat on my filesystem outside the $PATH. It was a trophy not a tool.

My Ruby monogamy was working out perfectly for me, whilst I collected futurist and novelty Ruby implementations as a passive hobby, until yesterday when I saw the above tweet.

That’s a bit of a bugger. I mean, why was he playing with Ruby 1.9.1 anyway?

Or perhaps having multiple version of Ruby on one computer isn’t some illicit activity. Everyone else is doing it, Your Honour.

I still didn’t care for having multiple ruby versions lying around in my path; but I had to fix rubigen. I just wanted a way to fix bugs against ruby 1.9+ and move on with my life. I didn’t want to have to do any manual work in order to set this up. Surely someone else has solved this problem already?

(more…)