Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle)

Posted by Dr Nic on April 01, 2008

Already know C extensions in RubyGems? Cool - then just run the following cmds and see what can be generated for you; plus check out the TextMate bundle at the bottom.

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test
rake test

For everyone else…

Its 15000km from Brisbane AU to Prague CZ where Euruko2008 - the European Ruby Conf - was held. I came ready to talk, to met lots of cool multi-lingual Rubyists, and to learn. Ooh, I learnt something alright.

Tim Becker was introducing Native C Extensions for Ruby, and fortunately he said “now, everyone, follow along with this example”. I’d never done native C extensions, but I’d received lots of requests from RubyGem developers on how to do it. I had no idea.

So I was typing in everything Tim told me to type in. When Tim changed slides too quickly, I may have yelled at him to slow down. Perhaps I was the only one doing his tutorial out of 300 people, but I didn’t care. This was gold.

After he finished his session, I dragged him off into the corridor with Jonas Pfenniger (zimbatm), and the three of us mapped out a generic layout for how native C extensions work within RubyGems. I didn’t know any of this, but Tim and Jonas did, and we probably looked silly sitting at a small table in the middle of the narrow corridor.

But at the end, we had a working RubyGem with native C extensions that were built: when the tests were executed via rake, and when the gem was installed. The next day I figured out how to get the C extension built via autotest.

Thanks to Tim and Jonas I was able to then write a extconf generator for RubyGems so that its super-super easy to get started writing native C extensions within RubyGems.

Tutorial

This tutorial is for *nix, as I’m still investigating win32 extensions, and jruby + .net/ironruby extensions. So when I figure that out - hopefully with the help of other people currently at RubyFools conferences, I’ll get back to you.

The code comes from the Pickaxe book - p264, and we’ll insert it into a new RubyGem using newgem (version 0.20.1+):

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test

Create a test for a class MyTest that doesn’t exist yet:

# test/test_my_test_extn.rb
require "test/unit"
require 'pickaxe'

class TestMyTestExtn < Test::Unit::TestCase
  def test_working
    t = MyTest.new
    assert_equal(Object, MyTest.superclass)
    assert_equal(MyTest, t.class)
    t.add(1)
    t.add(2)
    assert_equal([1,2], t.instance_eval("@arr"))
  end
end

Run rake to build the C extension and run the tests. You can also run autottest and it will automatically build the C extension before running the tests.

To create the MyTest class, using the code from p262 of the Pickaxe book:

#include "ruby.h"
static int id_push;

static VALUE t_init(VALUE self)
{
  VALUE arr;
  arr = rb_ary_new();
  rb_iv_set(self, "@arr", arr);
  return self;
}

static VALUE t_add(VALUE self, VALUE obj)
{
  VALUE arr;
  arr = rb_iv_get(self, "@arr");
  rb_funcall(arr, id_push, 1, obj);
  return arr;
}

VALUE cTest;
void Init_my_test() {
  cTest = rb_define_class("MyTest", rb_cObject);
  rb_define_method(cTest, "initialize", t_init, 0);
  rb_define_method(cTest, "add", t_add, 1);
  id_push = rb_intern("push");
}

To lib/pickaxe.rb:

require "my_test.so"
# or require 'my_test' if its unique

The last line will import the generated shared library. If the RubyGem is tested or installed on Windows, then the .dll file will be automatically loaded instead. The “.so” notation is merely a placeholder to explicitly specify the shared C-extension, rather than any Ruby library of the same name.

Now run tests (rake), the C extension will be rebuilt and the tests will pass.

Build and install RubyGem

rake manifest:refresh
rake install_gem
irb -rubygems -rpickaxe
> a = MyTest.new
> a.add 3

You have successfully created a C-extension within RubyGems, using TDD.

TextMate bundle for Ruby C extensions

I’ve started a TextMate bundle to give syntax highlighting + some simple snippets for developing the C files for Ruby extensions.

To install:

cd ~/Library/Application Support/TextMate/Bundles
git clone git://github.com/drnic/ruby-c-extensions-tmbundle.git "Ruby C Extensions.tmbundle"

or 

wget http://github.com/drnic/ruby-c-extensions-tmbundle/tarball/master
tar xfv drnic-ruby-c-extensions-tmbundle-master.tar.gz
mv drnic-ruby-c-extensions-tmbundle-master "Ruby C Extensions.tmbundle"

Then restart TextMate or “Reload Bundles”.

You can clone/fork the source via http://github.com/drnic/ruby-c-extensions-tmbundle/tree/master

Trackbacks

Use this link to trackback from your own site.

Comments

Leave a response

  1. Luis Lavena Wed, 02 Apr 2008 00:19:14 UTC

    Hey Dr.Nic!

    You don’t need to add the ‘.so’ indicating to require the extension, it will automatically look for it or a .rb file with the same name

    Worked without problems, with both versions of Ruby, Visual C 6 and MinGW.

    Thank you for the ping! ;-)

    Regards,
    Luis

  2. Urma Wed, 02 Apr 2008 00:25:55 UTC

    That LOAD_PATH thing is kinda odd. Are you sure it works when installed as an actual gem? In my experience the extension target always gets moved into lib/ after Rubygems builds it.

  3. Dr Nic Wed, 02 Apr 2008 17:13:24 UTC

    @Luis Lavena [via] - yeah, I kinda like explicitly stating that you’re loading the extension as it communicates to the read where they should find the source, I think. But that’s just imo.

    I’ll email you about how to pre-build mswin32 etc gems, and if extra rake tasks would be helpful.

  4. Dr Nic Wed, 02 Apr 2008 17:19:51 UTC

    @Urma [via] - yeah that looks to be true; so the LOAD_PATH bit needs to be in the test file still, but can be removed from the root lib file. I’ve removed the line about adding the $LOAD_PATH now; thx.

  5. [...] Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle) by Dr Nic [...]

  6. Rails Podcast Brasil - Episódio 12 Mon, 07 Apr 2008 23:30:47 UTC

    [...] Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle) [...]

  7. [...] Nic was kind enough to write a generator to handle native extensions for his newgem tool. newgem is definitely worth trying out, it generates a LOT of boiler plate, [...]

  8. Geoff Wed, 09 Apr 2008 11:52:31 UTC

    Thanks Dr. Nic, you continue to make my life easier :D

Comments