Dr Nic

My RubyGems development tools and workflow

The Open Source Developers Conference (osdc) is a nifty conference – people from different language communities, who all do open source, come together in harmony. They simultaneously snigger at each other and then proceed to steal each others ideas when they aren’t looking.

Miscellaneous idea possibly worth stealing from Perl: when you install Perl modules from CPAN it runs the module’s tests locally to confirm everything is oaky-dokey. You can manually do this via gem install --test rails, but its not the default. Nor have I ever used the --test flag. I just looked it up via gem help install. I like it.

Another CPAN thing to steal: For any RubyGem or perhaps Rails project on rubyforge/github/whatever, show the aggregate status of tests. Here’s a sample from CPAN. Lots of green lines but the aggregate change of this project working? 11%. I like the chart, and perhaps runcoderun or some other hosted CI service to generate a sexy graph of test outputs for dependencies.

Anyway, Ruby. I did a talk. I have slides.

Over time I’ve written a few RubyGems and am pretty happy with my basic tools and workflow for getting new gems out the door or maintaining existing projects. I use newgem + a patched version of hoe, I use git + github, and recently I started using runcoderun for hosted continuous integration.

If you’re new to creating your own RubyGems, perhaps my workflow and tools are a useful starting point to follow.

If you want an account with runcoderun, and I reckon you do, then hassle Rob Sanheim (twitter). For what its worth, tell him I said it was urgent.

There is now a sexy blog badge to show off the current pass/fail state of each of your projects, by Glenn Vanderburg. Badges are fun. It seems to clash with having github-badge in the same page, and there are other bugs with these badges. Probably my fault. I should investigate that soon.

newgem 1.0.0 all thanks to Cucumber

The New Gem Generator (newgem) was exciting, moderately revolutionary, and definitely helpful two years ago when I created it. Of late it seems to attract a chunk of criticism:

  • making a new gem, but newgem seems broken… hoe hoe
  • NewGem has the “hoe” virus. Much prefer Mr. Bones.
  • the newgem site is begging for someone to put the word fuck on its front page
  • Almost two days fighting with newgem, but today i won! The secret is hoe 1.7.0 and rubygem 1.2.0.
  • unfortunately for the one project I went with newgem, will give Mr. Bones a try on the next gem I throw out there.
  • sow or newgem, neither, Mr. Bones
  • egads, the website newgem sets up for you looks awful
  • newgem seems to be failed to generate package if AUTHOR is an array of authors.

On the positive side of the spectrum is the following list:

  • newgem? really? you rock drnic!

Comparatively, the two lists are awfully different in length. And not in a good way. No, not at all.

I know about these criticisms and platitude(s) because they appear publicly on newgem’s website within a live Twitter search of ‘newgem’. So that the twitter messages on the project’s own homepage are more positive, it was time for a new direction. A new beginning. It was time for a change.

So I fixed it. All of it. As of release 1.0.3 it is perfect [1].

To help you realise how wonderful newgem now is, I shall use the time-proven medium for proving awesomeness: a list. The bullet points are for free.

  • newgem now finally gone 1.0.0. It made it all the way to 0.29.0, but I think 1.0.0 was needed to transfer the message of a new beginning. This is the Obama of Gem Generators.
  • Generated gems are 50% smaller. No more config/hoe.rb. No more website folder (by default). No more tasks folder. No more license file. No more version.rb file.
  • Very little config required before releasing your gem. Just a few fields in the Rakefile.
  • You can use rspec or test/unit for unit testing (option: -T rspec)
  • You can use cucumber for functional testing (option: -i cucumber or run script/generate install_cucumber)
  • Generated gems are future-proofed. They will automatically benefit from future newgem releases.
  • Executable apps within gems now have a lib/appname/cli.rb file for the code base, and a lightweight bin/appname (option: -b appname or run script/generate executable appname)
  • Your README file can be called README.rdoc so it appears nicely formatted on github. No more hoe warning messages about “README.txt is missing” (see feature)
  • GitHub RubyGem support. rake gemspec generates a clean my_project.gemspec file that will work with GitHub
  • RubyForge support. As before, rake release VERSION=X.Y.Z releases your project to RubyForge (see ‘preparing for releases to rubyforge’ help page)
  • newgem’s website is a different colour. Its a nice peppermint green colour. The default website template is now this theme too.
  • You can raise bugs or suggest improvements via Lighthouse tracker

Installation

sudo gem install newgem

Usage

To create your RubyGem scaffold:

newgem mygem
newgem mygem -b myapp              # create a CLI executable
newgem mygem -T rspec -i cucumber  # use rspec and cucumber for gem tests
newgem mygem -w                    # create a simple website
newgem -h                          # get help

Now your code goes in lib folder, and your tests go in test, spec, and/or features as appropriate.

There are a bunch of rails-esque generators (like model or migration) that you can use to help your development:

script/generate executable myapp                       # create your own command-line interface (CLI)
script/generate extconf mylib                          # starting point for C-extensions, plus TDD framework
script/generate component_generator mygenerator scope  # create your own generators for Rails, Merb, RubyGems
script/generate application_generator myapp            # create a CLI that is a generator for something
script/generate -h                                     # get help

Bugs and suggestions

You can raise bugs or suggest improvements via Lighthouse tracker

Thanks goes to… Cucumber

Aside from several days of my time refactoring it, reducing it, and doubling the amount of awesomeness within it, all its wondefulness is thanks to Cucumber.

Cucumber is the successor to Rspec Story Runner. I never found time to play with Story Runner, but Cucumber is blowing my mind with awesomeness. My attention-span is short so I may be forgetting something but I think Cucumber could be the most important piece of software released in 2008 for Ruby-based developers.

Cucumber gave me a framework to specify newgem’s expected behaviour; its features. First I wrote feature descriptions for known, expected behaviour. Then I refactored the crap out of newgem until it was in tip-top shape.

There are over 90 feature steps defining newgem’s current features. To run them:

gem unpack newgem
cd newgem-*
sudo gem install cucumber
cucumber features

And watch the awesomeness of Cucumber unfold before your eyes. What you are seeing isn’t just newgem’s generators being executed, but also the generated code is being executed, rake tasks executed, and generated test files tested.

I can now setup continuous integration for newgem. I have a framework to know that newgem, or any other RubyGem, is doing what it should do.

UPDATE: I want to thank David Chelimsky for our time hanging out in Brazil during RailsSummit. I saw him using Cucumber, and talking about it on stage and then help helped me whilst I integrated it into newgem as a generator and then using it internally itself. For a day and a half we hung out in the hotel foyer. He’s so wonderful.

Cucumber makes me so happy.

Summary

Use newgem. Write gems.

Use mrbones. Write gems.

Use sow. Write gems.

Use gemhub. Write gems.

And write cucumber feature descriptions first. Then unit tests. Then code. Then release. Then profit.

[1] All claims of perfection are for the express purpose of making you try the product enough to use it, share it with your friends, and wrap it up and give it as a gift to family on Xmas day. Gift boxes are available upon request.

Unit Testing iPhone apps with Ruby: rbiphonetest

rbiphonetest logo

Everything to love about Ruby: the concise, powerful language; the sexy testing frameworks; and finally, the people.

Everything to love about Objective-C: hmmm; well…; and finally, its the only high-level language you can use to write iPhone apps.

On iPhone 2.0, to arrive on the 11th of July, you cannot run RubyCocoa. But you can run it on your Mac, so let’s use it to unit test your Objective-C classes. This tutorial shows you how to get started using a new project rbiphonetest [GitHub | Lighthouse | Google Group]

If you followed some of my recent tweets, this project was previously called “iphoneruby”. And alas, the screencast also calls it “iphoneruby” but that was a crap name. People thought it was a way to run Ruby on the iphone. I can’t do that yet. So, a far better name is ‘rbiphonetest’. [track on summize]

Even if you’ve never touched Objective-C, Cocoa, the iPhone SDK, nor RubyCocoa I recommend watching the video anyway. It should give you hope that if you make the transition to iPhone development you don’t have to go alone without Ruby: your trusty swiss army knife of language/libraries/tools.

The screencast is also available in high-def video (55Mb QuickTime)


Unit Testing iPhone apps using Ruby from Dr Nic on Vimeo.

Installation and Usage

To summarize the video, but change ‘iphoneruby’ to ‘rbiphonetest’, you install the framework via RubyGems:

sudo gem install rbiphonetest

Then change to your project’s folder and install the test framework:

rbiphonetest .

Finally, for each generic, non-UIKit-framework-using class you want to test:

script/generate model WidgetModel

Then write your tests in test/test_widget_model.rb

Supported Cocoa & iPhone frameworks

The mysterious, magical premise upon which rbiphonetest depends is possibly erroneous: that your Objective-C class can be compiled and tested against your OS X/Intel frameworks, and if your tests pass you assume you can then compile and include your class with the the iPhone/ARM frameworks.

I’m willing to go with this assumption until its proven dangerously flawed by some angry 20-year veteran of NextStep/Cocoa/iPhone. But really, how different could NSString be on the iPhone versus your Mac?

Fortunately there is one way to check for significant differences between your available Mac-based frameworks, such as Cocoa, and the iPhone-based frameworks, such as UIKit. We need to compare the framework names, header files and method signatures.

So for example, you cannot currently unit test any class that depends on/includes the UIKit framework. Why? It doesn’t exist on your Mac, so the Mac/Intel compiler cannot link it in. We’re compiling and running our tests with RubyCocoa, which itself is built against the Mac/Intel frameworks, not the iPhone frameworks. Hell, Laurent doesn’t even own an iPhone :) [Laurent is the Apple-employee maintainer of RubyCocoa and the newer MacRuby]

Similarly, its no use including/linking the Cocoa framework into your Objective-C class. Why? It doesn’t exist on the iPhone. It has its own UI frameworks, collectively called ‘UIKit’.

So for the moment we cannot test UI-related, iPhone-API-specific code. But we can test generic Objective-C. That’s better than a kick in the teeth. Surely. I mean, in the teeth… that’d friggin’ hurt.

“Fair enough Dr Nic, so which frameworks can my code use and yet still unit test it with your oh-so-special test library thingy?” Keep your pants on, I’m getting there. [ref]

To the best of my ability, I’ve compared the two sets of frameworks and listed the available Frameworks that are available on both the iPhone and your Mac. There are about a dozen. The most important is called ‘Foundation’. It holds gold nuggets like ‘NSString’.

The list of platform differences is on the wiki as a reference.

Note, this list doesn’t guarantee that any two framework classes – the iPhone and matching Mac framework – will behave the same. This list is compiled by finding the set of Frameworks with the same name on both platforms, e.g. Foundation.

Then it compares the set of public header files (Foundation.framework/Headers/*.h files) This comparison is by method signature. It pulls all lines from each header that start with + or – (+ is a class method and – is an instance method in Objective-C) and compares the two lists. If there is a single difference in the method signatures of the header files in the two platforms it is marked on the wiki page. You’ll need to look at the two header files yourself to see the differences. Some header files are ugly. C-based anything starts ugly and goes down from there, I think.

Python testing of iPhone Objective-C?

In the Python world there is PyObjC, a bridge-based twin to RubyCocoa. If you are a Python developer you could easily port this project to use PyObjC I would think. Ping me if you are attempting this and need any help.

Summary

I think this project can give Ruby developers a happy place to work from as they write their Objective-C/iPhone code. You still need to wire up your UI views and controller classes manually, but if you push all the “oooh that code really needs some tests” classes away from the UI-dependent frameworks then you can hook it up to rbiphonetest and write your tests in Ruby.

Currently the generator creates test/unit test stubs. I personally then add the Shoulda gem into my test_helper.rb for my apps. If an rspec and/or test/spec developer can help with adding support to the generators I’m certain the large rspec user-base would be happy campers.

Similarly, someone might like to investigate using MacRuby to run the tests instead of RubyCocoa. Fast tests vs slow tests. You choose.

What the?

Sometimes I re-read what I’ve written and notice things that don’t seem to make sense, but are in my vocabulary nonetheless. Yep, the things you learn living in Australia.

“Keep your pants on” – this seems to imply that until I mentioned otherwise you were about to take your pants off. Hardly relevant at any stage during this article, we’d both agree. Most code-based blog articles are “pants on”. This phrase means “don’t get upset”. You can try to figure out how you go from “don’t get upset” to “keep your pants on”. I have no idea.

Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle)

Already know C extensions in RubyGems? Cool – then just run the following cmds and see what can be generated for you; plus check out the TextMate bundle at the bottom.

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test
rake test

For everyone else…

Its 15000km from Brisbane AU to Prague CZ where Euruko2008 – the European Ruby Conf – was held. I came ready to talk, to met lots of cool multi-lingual Rubyists, and to learn. Ooh, I learnt something alright.

Tim Becker was introducing Native C Extensions for Ruby, and fortunately he said “now, everyone, follow along with this example”. I’d never done native C extensions, but I’d received lots of requests from RubyGem developers on how to do it. I had no idea.

So I was typing in everything Tim told me to type in. When Tim changed slides too quickly, I may have yelled at him to slow down. Perhaps I was the only one doing his tutorial out of 300 people, but I didn’t care. This was gold.

After he finished his session, I dragged him off into the corridor with Jonas Pfenniger (zimbatm), and the three of us mapped out a generic layout for how native C extensions work within RubyGems. I didn’t know any of this, but Tim and Jonas did, and we probably looked silly sitting at a small table in the middle of the narrow corridor.

But at the end, we had a working RubyGem with native C extensions that were built: when the tests were executed via rake, and when the gem was installed. The next day I figured out how to get the C extension built via autotest.

Thanks to Tim and Jonas I was able to then write a extconf generator for RubyGems so that its super-super easy to get started writing native C extensions within RubyGems.

Tutorial

This tutorial is for *nix, as I’m still investigating win32 extensions, and jruby + .net/ironruby extensions. So when I figure that out – hopefully with the help of other people currently at RubyFools conferences, I’ll get back to you.

The code comes from the Pickaxe book – p264, and we’ll insert it into a new RubyGem using newgem (version 0.20.1+):

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test

Create a test for a class MyTest that doesn’t exist yet:

# test/test_my_test_extn.rb
require "test/unit"
require 'pickaxe'

class TestMyTestExtn < Test::Unit::TestCase
  def test_working
    t = MyTest.new
    assert_equal(Object, MyTest.superclass)
    assert_equal(MyTest, t.class)
    t.add(1)
    t.add(2)
    assert_equal([1,2], t.instance_eval("@arr"))
  end
end

Run rake to build the C extension and run the tests. You can also run autottest and it will automatically build the C extension before running the tests.

To create the MyTest class, using the code from p262 of the Pickaxe book:

#include "ruby.h"
static int id_push;

static VALUE t_init(VALUE self)
{
  VALUE arr;
  arr = rb_ary_new();
  rb_iv_set(self, "@arr", arr);
  return self;
}

static VALUE t_add(VALUE self, VALUE obj)
{
  VALUE arr;
  arr = rb_iv_get(self, "@arr");
  rb_funcall(arr, id_push, 1, obj);
  return arr;
}

VALUE cTest;
void Init_my_test() {
  cTest = rb_define_class("MyTest", rb_cObject);
  rb_define_method(cTest, "initialize", t_init, 0);
  rb_define_method(cTest, "add", t_add, 1);
  id_push = rb_intern("push");
}

To lib/pickaxe.rb:

require "my_test.so"
# or require 'my_test' if its unique

The last line will import the generated shared library. If the RubyGem is tested or installed on Windows, then the .dll file will be automatically loaded instead. The “.so” notation is merely a placeholder to explicitly specify the shared C-extension, rather than any Ruby library of the same name.

Now run tests (rake), the C extension will be rebuilt and the tests will pass.

Build and install RubyGem

rake manifest:refresh
rake install_gem
irb -rubygems -rpickaxe
> a = MyTest.new
> a.add 3

You have successfully created a C-extension within RubyGems, using TDD.

TextMate bundle for Ruby C extensions

I’ve started a TextMate bundle to give syntax highlighting + some simple snippets for developing the C files for Ruby extensions.

To install:

cd ~/Library/Application Support/TextMate/Bundles
git clone git://github.com/drnic/ruby-c-extensions-tmbundle.git "Ruby C Extensions.tmbundle"

or 

wget http://github.com/drnic/ruby-c-extensions-tmbundle/tarball/master
tar xfv drnic-ruby-c-extensions-tmbundle-master.tar.gz
mv drnic-ruby-c-extensions-tmbundle-master "Ruby C Extensions.tmbundle"

Then restart TextMate or “Reload Bundles”.

You can clone/fork the source via http://github.com/drnic/ruby-c-extensions-tmbundle/tree/master

RubiGen video from RejectConf Berlin

Go straight to video below

I took a video recorder to the RejectConf, like I did in Portland. Unfortunately, there were two reasons I didn’t record any of them.

Firstly, there didn’t seem to be any obvious place to position the camera.

Secondly, it was deemed critical that everyone does their talks in the dark. The conference isn’t run in the dark, local Ruby groups don’t run meetings in the dark but consecutive RejectConfs have been run by adminstrators with a dark fetish. Great for drinking beer and heckling presenters. Bad for video recording.

The makers of the JVC HDD camcorder – a nifty device with a 20G HDD in it – don’t make it possible to record in the dark. Not because it doesn’t have a “night time mode” – apparently it does – but when you are already in the dark, you can’t figure out how to turn it on.

Ok, fine, if I’d made an effort I could have figured it out. So, let’s use excuse #1 as the reason for not recording the presentations.

Its a valid excuse, the Pirate Cove was chock-a-block full of people. The local Berlin Ruby Group did an awesome job of finding a great “underground”-esque venue.

Fortunately a fellow Australian – Marcus Crafter – had a front row position, and a MacBook Pro. With said device, he captured my talk on RubiGen (and John Barton’s).

RejectConf video of RubiGen

In 5 minutes, I make a Merb generator, using RubiGen and NewGem. Nifty stuff indeed.

Recorded and published by Marcus Crafter

There were lots of other awesome presentations (that is perhaps a dubious inference that mine was awesome), but it was dark, I had beer in both hands, and I was too busy yelling “AUSTRALIA!!!” to write notes.

What a great night :)