Already know C extensions in RubyGems? Cool - then just run the following cmds and see what can be generated for you; plus check out the TextMate bundle at the bottom.

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test
rake test

For everyone else…

Its 15000km from Brisbane AU to Prague CZ where Euruko2008 - the European Ruby Conf - was held. I came ready to talk, to met lots of cool multi-lingual Rubyists, and to learn. Ooh, I learnt something alright.

Tim Becker was introducing Native C Extensions for Ruby, and fortunately he said “now, everyone, follow along with this example”. I’d never done native C extensions, but I’d received lots of requests from RubyGem developers on how to do it. I had no idea.

So I was typing in everything Tim told me to type in. When Tim changed slides too quickly, I may have yelled at him to slow down. Perhaps I was the only one doing his tutorial out of 300 people, but I didn’t care. This was gold.

After he finished his session, I dragged him off into the corridor with Jonas Pfenniger (zimbatm), and the three of us mapped out a generic layout for how native C extensions work within RubyGems. I didn’t know any of this, but Tim and Jonas did, and we probably looked silly sitting at a small table in the middle of the narrow corridor.

But at the end, we had a working RubyGem with native C extensions that were built: when the tests were executed via rake, and when the gem was installed. The next day I figured out how to get the C extension built via autotest.

Thanks to Tim and Jonas I was able to then write a extconf generator for RubyGems so that its super-super easy to get started writing native C extensions within RubyGems.

Tutorial

This tutorial is for *nix, as I’m still investigating win32 extensions, and jruby + .net/ironruby extensions. So when I figure that out - hopefully with the help of other people currently at RubyFools conferences, I’ll get back to you.

The code comes from the Pickaxe book - p264, and we’ll insert it into a new RubyGem using newgem (version 0.20.1+):

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test

Create a test for a class MyTest that doesn’t exist yet:

# test/test_my_test_extn.rb
require "test/unit"
require 'pickaxe'

class TestMyTestExtn < Test::Unit::TestCase
  def test_working
    t = MyTest.new
    assert_equal(Object, MyTest.superclass)
    assert_equal(MyTest, t.class)
    t.add(1)
    t.add(2)
    assert_equal([1,2], t.instance_eval(”@arr”))
  end
end

Run rake to build the C extension and run the tests. You can also run autottest and it will automatically build the C extension before running the tests.

To create the MyTest class, using the code from p262 of the Pickaxe book:

#include "ruby.h"
static int id_push;

static VALUE t_init(VALUE self)
{
  VALUE arr;
  arr = rb_ary_new();
  rb_iv_set(self, "@arr", arr);
  return self;
}

static VALUE t_add(VALUE self, VALUE obj)
{
  VALUE arr;
  arr = rb_iv_get(self, "@arr");
  rb_funcall(arr, id_push, 1, obj);
  return arr;
}

VALUE cTest;
void Init_my_test() {
  cTest = rb_define_class("MyTest", rb_cObject);
  rb_define_method(cTest, "initialize", t_init, 0);
  rb_define_method(cTest, "add", t_add, 1);
  id_push = rb_intern("push");
}

To lib/pickaxe.rb:

require "my_test.so"
# or require 'my_test' if its unique

The last line will import the generated shared library. If the RubyGem is tested or installed on Windows, then the .dll file will be automatically loaded instead. The “.so” notation is merely a placeholder to explicitly specify the shared C-extension, rather than any Ruby library of the same name.

Now run tests (rake), the C extension will be rebuilt and the tests will pass.

Build and install RubyGem

rake manifest:refresh
rake install_gem
irb -rubygems -rpickaxe
> a = MyTest.new
> a.add 3

You have successfully created a C-extension within RubyGems, using TDD.

TextMate bundle for Ruby C extensions

I’ve started a TextMate bundle to give syntax highlighting + some simple snippets for developing the C files for Ruby extensions.

To install:

cd ~/Library/Application Support/TextMate/Bundles
git clone git://github.com/drnic/ruby-c-extensions-tmbundle.git "Ruby C Extensions.tmbundle"

or 

wget http://github.com/drnic/ruby-c-extensions-tmbundle/tarball/master
tar xfv drnic-ruby-c-extensions-tmbundle-master.tar.gz
mv drnic-ruby-c-extensions-tmbundle-master "Ruby C Extensions.tmbundle"

Then restart TextMate or “Reload Bundles”.

You can clone/fork the source via http://github.com/drnic/ruby-c-extensions-tmbundle/tree/master

Go straight to video below

I took a video recorder to the RejectConf, like I did in Portland. Unfortunately, there were two reasons I didn’t record any of them.

Firstly, there didn’t seem to be any obvious place to position the camera.

Secondly, it was deemed critical that everyone does their talks in the dark. The conference isn’t run in the dark, local Ruby groups don’t run meetings in the dark but consecutive RejectConfs have been run by adminstrators with a dark fetish. Great for drinking beer and heckling presenters. Bad for video recording.

The makers of the JVC HDD camcorder - a nifty device with a 20G HDD in it - don’t make it possible to record in the dark. Not because it doesn’t have a “night time mode” - apparently it does - but when you are already in the dark, you can’t figure out how to turn it on.

Ok, fine, if I’d made an effort I could have figured it out. So, let’s use excuse #1 as the reason for not recording the presentations.

Its a valid excuse, the Pirate Cove was chock-a-block full of people. The local Berlin Ruby Group did an awesome job of finding a great “underground”-esque venue.

Fortunately a fellow Australian - Marcus Crafter - had a front row position, and a MacBook Pro. With said device, he captured my talk on RubiGen (and John Barton’s).

RejectConf video of RubiGen

In 5 minutes, I make a Merb generator, using RubiGen and NewGem. Nifty stuff indeed.

There were lots of other awesome presentations (that is perhaps a dubious inference that mine was awesome), but it was dark, I had beer in both hands, and I was too busy yelling “AUSTRALIA!!!” to write notes.

What a great night

I don’t really talk about my projects after I release them except to show off fancy new things, like newgem sporting the new RubiGen generator.

So I don’t know why I give updates here about one of the smallest projects - map_by_method.

Probably, its because it doesn’t deserve a Google Group or even a webpage really. It should probably be integrated into activesupport gem or something.

Except, it just never seemed to 100% work.

That is, it go more blog coverage than anything else, but wasn’t always useful. Yeah, a like Miss South Carolina.

BUT Version 0.8.2 is now out. It fixes - I believe - all known problems - I think.

gem install map_by_method

In your code (or config/environment.rb for rails):

gem 'map_by_method', '>=0.8.2'
require 'map_by_method'

I added a few more iterator methods too: sort_by, group_by, and index_by.

>> Conference.find(1).conference_sessions.group_by_from
=> returns all MyConfPlan conference sessions grouped by start time, for RailsConf2007

MyConfPlan…

Oh yes it sold. Announcement coming soon.

Plus, a link to the code - as it is being open sourced by its new owner. Sweet!

Well… sweet for you. I already had access to the code.

Next on the todo list

Add something similar to Ambition by Err “we-write-so-many-cool-projects-I-don’t-know-how-we-find-time-to-maintain-them-all” the Blog, Chris and PJ.

I’d kind link something like:

Conference.select { |c| c =~ "Rails").sort_by_name

Which would behave like:

>> Conference.select {|c| c.name =~ /Rails/}.sort_by(&:name).to_sql
=> "SELECT * FROM conferences WHERE conferences.\"name\" ~ 'Rails' ORDER BY conferences.name"

Oh well, perhaps we can talk about it during RailsConf Europe, when I get the microphone to talk about such tom-foolery and hi-jinx: Meta-Magic in Rails: Become a Master Magician

The New Gem Generator (0.13.0)’s newgem command now behaves like the rails command thanks to RubiGen - a new project that is an extraction of the rails_generator.

Developing a RubyGem? You can now have a script/generate command, just like Rails. Other projects, say like Merb, can do this too.

Awesomeness with a Gold Star. (disclosure: Dr Nic awarded this Gold Star to himself)

Warning

This article can get confusing.

Especially around the heading “Generating Generators”.

Plus, this article talks about two projects - an update to New Gem Generator (0.13.0) and the new RubiGen project.

So, give yourself time to digest it all, and then perhaps come back and read it again.

And then perhaps re-write the article for me.

“A generator that can provide generators to your projects, so that you can write generators for your other projects.”

Re-write it - I dare you.

Background

One of the killer features of Rails is the Rails Generator.

It does three things:

• generates an entire scaffold for your application, thus sharing with you its conventions (over configuration) methodology; and
• it then allows you to generate more stuff, like models, controllers, plugins etc. Things that are relevant to a Rails app. Finally,
• it allows you to write your own generators for Rails apps.

Now you can do all this with New Gem.

Killer Feature #1 - upgrade path

Never before have you been able to run ‘newgem’ on top of an existing RubyGem. Why? It used to blow away the entire folder, and then start writing new files… not very friendly.

Now, just like the rails command, you can go into your RubyGem development directory, and run newgem . and you will be asked which files to override.

NOTE: Copy your Rakefile to Rakefile.old, and after running newgem . copy the configuration information into the new file config/hoe.rb.

If you’re using newgem already, I dare you to upgrade as above. (Ok, perhaps make a backup copy of your work first…)

Killer Feature #2 - script/generate

Once you’ve upgraded, or create a new RubyGem (using newgem gemname), you’ll now have two scripts: script/generate and script/destroy, just like Rails has. (and matching generate.cmd and destroy.cmd for Windows users)

Try them out:

script/generate
...
Installed Generators
  Rubygems: application_generator, component_generator, executable, install_jruby, install_rspec, install_website
  Builtin: test_unit

Oh yeah, I’ve had some fun extracting things into generators.

• Install the dubious-looking NewGem website - script/generate install_website.
• Install RSpec support - script/generate install_rspec.
• Make the RubyGem a JRuby gem - script/generate install_jruby (the generated gem will have -jruby.gem in its name)
• Create an executable Ruby app - script/generate executable appname

These generators are also reused via the newgem commands various options (Run newgem to see them.)

Killer Feature #3 - generate generators

I’ll post more about RubiGen later, but you can create a new rails-like command-line app that generates a whole stack load of directories and files using script/generate application_generator appname.

There is a large USAGE rundown if you run script/generate application_generator.

Want to create your own generators for developing RubyGems? (similar to creating a generator for a Rails app, but for RubyGems)

script/generate component_generator foobar rubygems
    create  rubygems_generators/foobar/templates
    exists  test
    create  rubygems_generators/foobar/foobar_generator.rb
    create  test/test_foobar_generator.rb
    create  test/test_generator_helper.rb
    create  rubygems_generators/foobar/USAGE
    readme  readme

Firstly, note test/test_generatorname_generator.rb - that’s right, you get a test stub for your new generator. Start there, write tests, then write your generator. There’s inline help for useful assertions.

Secondly, note rubygems_generators folder. This folder is the “scope” of the generator. As it starts with “rubygems” the generator will only be available when you are developing rubygems. It will not show up in Rails, nor Merb or Camping or any other place that may support RubiGen one day.

Similarly, if you want to write a Rails generator using the component_generator then specify the scope as rails.

script/generate component_generator booya rails
     create  rails_generators/booya/templates
     exists  test
     create  rails_generators/booya/booya_generator.rb
     create  test/test_booya_generator.rb
  identical  test/test_generator_helper.rb
     create  rails_generators/booya/USAGE
     readme  readme

In Edge Rails (and any Rails version after 1.2.3), the script/generate command in Rails will search all RubyGems for /rails_generator/* folders in addition to the existing search paths (ticket).

Thanks goes to Rails Generator

I’ve long been in love with the Rails Generator for what it does, the beautiful syntax for specifying a generator, etc. I don’t know who wrote what bit, e.g. if DHH wrote all of it, or others wrote nice chunks, but its awesome. Thanks DHH and co. Thanks to Jeremy Kemper

RubiGen is 95% Rails Generator code, with extensions to support scoping. Setting Rails Generator free of its Rails constraints is a tribute to it.

Now RubyGems can use generators, and any other frameworks can integrate generators and their developers can write and distribute additional generators for those frameworks.

I award myself 2 gold stars, and a scratch-n-sniff

I was always annoyed that map_by_method was broken for ActiveRecord has_many associations. 6 mths later I finally fixed it.

That’s the magic of Open Source Software. [/end sarcasm]

So now, the following example works like it should:

$ gem install map_by_method
$ console
> require 'map_by_method'  # stick this in your environment.rb for Rails
> user = User.find_by_name "Dr Nic"
> user.companies.map_by_name
=> ['Dr Nic Academy', 'Dr Nic Institute of Being Silly']
> user.companies.map_by_id_and_name
=> [[1, 'Dr Nic Academy'], [9, 'Dr Nic Institute of Being Silly']]

Recap: why use map_by_method?

Try the following example:

> user.companies.map_by_employees.flatten
=> list of all employees of user

Versus:

> user.companies.map { |company| company.employees}.flatten
or
> user.companies.map(&:employees).flatten

Or compare:

> user.companies.map_by_id_and_name
=> [[1, 'Dr Nic Academy'], [9, 'Dr Nic Institute of Being Silly']]

Versus:

> user.companies.map { |company| [company.id, company.name]}

That is, it looks and feels just like ActiveRecord’s #find method, with its find_by_first_name_and_last_name magic.

Summary

No {, }, |, &, or : required. Just clean method names.

Bonus other gem

In the spirit of ActiveRecord hacks, there is to_activerecord:

$ gem install to_activerecord
$ console
> require 'to_activerecord'  # stick this in your environment.rb for Rails
> [1,2,3].to_user
=> [list of User with id's 1,2,3]

To me, this suffix operator reads cleaner than the traditional:

> User.find([1,2,3])

For example, if you want to perform an operation on the list of Users:

> ids = [1,2,3]
> ids.to_user.map_by_name
=> ['Dr Nic', 'Banjo', 'Nancy']

Versus:

> User.find(ids).map_by_name