Dr Nic

Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle)

Already know C extensions in RubyGems? Cool – then just run the following cmds and see what can be generated for you; plus check out the TextMate bundle at the bottom.

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test
rake test

For everyone else…

Its 15000km from Brisbane AU to Prague CZ where Euruko2008 – the European Ruby Conf – was held. I came ready to talk, to met lots of cool multi-lingual Rubyists, and to learn. Ooh, I learnt something alright.

Tim Becker was introducing Native C Extensions for Ruby, and fortunately he said “now, everyone, follow along with this example”. I’d never done native C extensions, but I’d received lots of requests from RubyGem developers on how to do it. I had no idea.

So I was typing in everything Tim told me to type in. When Tim changed slides too quickly, I may have yelled at him to slow down. Perhaps I was the only one doing his tutorial out of 300 people, but I didn’t care. This was gold.

After he finished his session, I dragged him off into the corridor with Jonas Pfenniger (zimbatm), and the three of us mapped out a generic layout for how native C extensions work within RubyGems. I didn’t know any of this, but Tim and Jonas did, and we probably looked silly sitting at a small table in the middle of the narrow corridor.

But at the end, we had a working RubyGem with native C extensions that were built: when the tests were executed via rake, and when the gem was installed. The next day I figured out how to get the C extension built via autotest.

Thanks to Tim and Jonas I was able to then write a extconf generator for RubyGems so that its super-super easy to get started writing native C extensions within RubyGems.

Tutorial

This tutorial is for *nix, as I’m still investigating win32 extensions, and jruby + .net/ironruby extensions. So when I figure that out – hopefully with the help of other people currently at RubyFools conferences, I’ll get back to you.

The code comes from the Pickaxe book – p264, and we’ll insert it into a new RubyGem using newgem (version 0.20.1+):

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test

Create a test for a class MyTest that doesn’t exist yet:

# test/test_my_test_extn.rb
require "test/unit"
require 'pickaxe'

class TestMyTestExtn < Test::Unit::TestCase
  def test_working
    t = MyTest.new
    assert_equal(Object, MyTest.superclass)
    assert_equal(MyTest, t.class)
    t.add(1)
    t.add(2)
    assert_equal([1,2], t.instance_eval("@arr"))
  end
end

Run rake to build the C extension and run the tests. You can also run autottest and it will automatically build the C extension before running the tests.

To create the MyTest class, using the code from p262 of the Pickaxe book:

#include "ruby.h"
static int id_push;

static VALUE t_init(VALUE self)
{
  VALUE arr;
  arr = rb_ary_new();
  rb_iv_set(self, "@arr", arr);
  return self;
}

static VALUE t_add(VALUE self, VALUE obj)
{
  VALUE arr;
  arr = rb_iv_get(self, "@arr");
  rb_funcall(arr, id_push, 1, obj);
  return arr;
}

VALUE cTest;
void Init_my_test() {
  cTest = rb_define_class("MyTest", rb_cObject);
  rb_define_method(cTest, "initialize", t_init, 0);
  rb_define_method(cTest, "add", t_add, 1);
  id_push = rb_intern("push");
}

To lib/pickaxe.rb:

require "my_test.so"
# or require 'my_test' if its unique

The last line will import the generated shared library. If the RubyGem is tested or installed on Windows, then the .dll file will be automatically loaded instead. The “.so” notation is merely a placeholder to explicitly specify the shared C-extension, rather than any Ruby library of the same name.

Now run tests (rake), the C extension will be rebuilt and the tests will pass.

Build and install RubyGem

rake manifest:refresh
rake install_gem
irb -rubygems -rpickaxe
> a = MyTest.new
> a.add 3

You have successfully created a C-extension within RubyGems, using TDD.

TextMate bundle for Ruby C extensions

I’ve started a TextMate bundle to give syntax highlighting + some simple snippets for developing the C files for Ruby extensions.

To install:

cd ~/Library/Application Support/TextMate/Bundles
git clone git://github.com/drnic/ruby-c-extensions-tmbundle.git "Ruby C Extensions.tmbundle"

or 

wget http://github.com/drnic/ruby-c-extensions-tmbundle/tarball/master
tar xfv drnic-ruby-c-extensions-tmbundle-master.tar.gz
mv drnic-ruby-c-extensions-tmbundle-master "Ruby C Extensions.tmbundle"

Then restart TextMate or “Reload Bundles”.

You can clone/fork the source via http://github.com/drnic/ruby-c-extensions-tmbundle/tree/master

Related posts:

  1. newgem 1.0.0 all thanks to Cucumber The New Gem Generator (newgem) was exciting, moderately revolutionary, and...
  2. TextMate bundles for Merb If you are using TextMate (OS X) or E Text...
  3. NewGem Generator – now with script/generate The New Gem Generator (0.13.0)’s newgem command now behaves like...
  4. Post-Halloween RadRails trick – all TextMate snippets available HTML snippets 48 Ruby snippets 199 I tease myself...

11 Responses to “Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle)”

  1. Luis Lavena says:

    Hey Dr.Nic!

    You don’t need to add the ‘.so’ indicating to require the extension, it will automatically look for it or a .rb file with the same name

    Worked without problems, with both versions of Ruby, Visual C 6 and MinGW.

    Thank you for the ping! ;-)

    Regards,
    Luis

  2. Urma says:

    That LOAD_PATH thing is kinda odd. Are you sure it works when installed as an actual gem? In my experience the extension target always gets moved into lib/ after Rubygems builds it.

  3. Dr Nic says:

    @Luis Lavena [via] – yeah, I kinda like explicitly stating that you’re loading the extension as it communicates to the read where they should find the source, I think. But that’s just imo.

    I’ll email you about how to pre-build mswin32 etc gems, and if extra rake tasks would be helpful.

  4. Dr Nic says:

    @Urma [via] – yeah that looks to be true; so the LOAD_PATH bit needs to be in the test file still, but can be removed from the root lib file. I’ve removed the line about adding the $LOAD_PATH now; thx.

  5. [...] Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle) by Dr Nic [...]

  6. [...] Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle) [...]

  7. [...] Nic was kind enough to write a generator to handle native extensions for his newgem tool. newgem is definitely worth trying out, it generates a LOT of boiler plate, [...]

  8. Geoff says:

    Thanks Dr. Nic, you continue to make my life easier :D

  9. [...] so cut me *some* slack, > will you. :P ) You should take a look at what Dr.Nic Williams did: http://drnicwilliams.com/2008/04/01/…s-in-rubygems/ > > | > | Why not compile natively on Windows? > > Setting up a MinGW/MSYS [...]

  10. Ram Singla says:

    For Windows the newgem commands through the following error
    can’t convert nil into String
    c:/ruby/lib/ruby/gems/1.8/gems/newgem-1.2.3/app_generators/newgem/newgem_generator.rb:169:in `join’

    You need to have ENV['HOME'] setup

    one way you can do is

    set HOME= %USERPROFILE%

    or you can go to

    My Computer -> Properties -> Advanced -> Environmental Variables

    and create a new User Variable

    with name HOME and with value %USERPROFILE%

    or best will be hoe library should fallback to

    ENV['USERPROFILE'] if ENV['HOME'] is nil [Which is the case for Windows]

  11. valley says:

    If followed all steps for my own project with the latest newgem.
    Everything works fine and it also installs the gem, but when i want
    to test it i always get the error ‘no such file to load — myproject’ in
    the lib/myproject.rb, where i tried

    require ‘myproject.so’ and require ‘myproject’

    I included the file in the Manifest with
    ext/myproject/myproject.o

    Did anything change in newgem since this article or why doesn’t it find the library anymore ?
    I run os x leopard with ruby 1.8.6

    Thanks for your help.
    Regards
    valley