Dr Nic

newgem 1.0.0 all thanks to Cucumber

The New Gem Generator (newgem) was exciting, moderately revolutionary, and definitely helpful two years ago when I created it. Of late it seems to attract a chunk of criticism:

  • making a new gem, but newgem seems broken… hoe hoe
  • NewGem has the “hoe” virus. Much prefer Mr. Bones.
  • the newgem site is begging for someone to put the word fuck on its front page
  • Almost two days fighting with newgem, but today i won! The secret is hoe 1.7.0 and rubygem 1.2.0.
  • unfortunately for the one project I went with newgem, will give Mr. Bones a try on the next gem I throw out there.
  • sow or newgem, neither, Mr. Bones
  • egads, the website newgem sets up for you looks awful
  • newgem seems to be failed to generate package if AUTHOR is an array of authors.

On the positive side of the spectrum is the following list:

  • newgem? really? you rock drnic!

Comparatively, the two lists are awfully different in length. And not in a good way. No, not at all.

I know about these criticisms and platitude(s) because they appear publicly on newgem’s website within a live Twitter search of ‘newgem’. So that the twitter messages on the project’s own homepage are more positive, it was time for a new direction. A new beginning. It was time for a change.

So I fixed it. All of it. As of release 1.0.3 it is perfect [1].

To help you realise how wonderful newgem now is, I shall use the time-proven medium for proving awesomeness: a list. The bullet points are for free.

  • newgem now finally gone 1.0.0. It made it all the way to 0.29.0, but I think 1.0.0 was needed to transfer the message of a new beginning. This is the Obama of Gem Generators.
  • Generated gems are 50% smaller. No more config/hoe.rb. No more website folder (by default). No more tasks folder. No more license file. No more version.rb file.
  • Very little config required before releasing your gem. Just a few fields in the Rakefile.
  • You can use rspec or test/unit for unit testing (option: -T rspec)
  • You can use cucumber for functional testing (option: -i cucumber or run script/generate install_cucumber)
  • Generated gems are future-proofed. They will automatically benefit from future newgem releases.
  • Executable apps within gems now have a lib/appname/cli.rb file for the code base, and a lightweight bin/appname (option: -b appname or run script/generate executable appname)
  • Your README file can be called README.rdoc so it appears nicely formatted on github. No more hoe warning messages about “README.txt is missing” (see feature)
  • GitHub RubyGem support. rake gemspec generates a clean my_project.gemspec file that will work with GitHub
  • RubyForge support. As before, rake release VERSION=X.Y.Z releases your project to RubyForge (see ‘preparing for releases to rubyforge’ help page)
  • newgem’s website is a different colour. Its a nice peppermint green colour. The default website template is now this theme too.
  • You can raise bugs or suggest improvements via Lighthouse tracker

Installation

sudo gem install newgem

Usage

To create your RubyGem scaffold:

newgem mygem
newgem mygem -b myapp              # create a CLI executable
newgem mygem -T rspec -i cucumber  # use rspec and cucumber for gem tests
newgem mygem -w                    # create a simple website
newgem -h                          # get help

Now your code goes in lib folder, and your tests go in test, spec, and/or features as appropriate.

There are a bunch of rails-esque generators (like model or migration) that you can use to help your development:

script/generate executable myapp                       # create your own command-line interface (CLI)
script/generate extconf mylib                          # starting point for C-extensions, plus TDD framework
script/generate component_generator mygenerator scope  # create your own generators for Rails, Merb, RubyGems
script/generate application_generator myapp            # create a CLI that is a generator for something
script/generate -h                                     # get help

Bugs and suggestions

You can raise bugs or suggest improvements via Lighthouse tracker

Thanks goes to… Cucumber

Aside from several days of my time refactoring it, reducing it, and doubling the amount of awesomeness within it, all its wondefulness is thanks to Cucumber.

Cucumber is the successor to Rspec Story Runner. I never found time to play with Story Runner, but Cucumber is blowing my mind with awesomeness. My attention-span is short so I may be forgetting something but I think Cucumber could be the most important piece of software released in 2008 for Ruby-based developers.

Cucumber gave me a framework to specify newgem’s expected behaviour; its features. First I wrote feature descriptions for known, expected behaviour. Then I refactored the crap out of newgem until it was in tip-top shape.

There are over 90 feature steps defining newgem’s current features. To run them:

gem unpack newgem
cd newgem-*
sudo gem install cucumber
cucumber features

And watch the awesomeness of Cucumber unfold before your eyes. What you are seeing isn’t just newgem’s generators being executed, but also the generated code is being executed, rake tasks executed, and generated test files tested.

I can now setup continuous integration for newgem. I have a framework to know that newgem, or any other RubyGem, is doing what it should do.

UPDATE: I want to thank David Chelimsky for our time hanging out in Brazil during RailsSummit. I saw him using Cucumber, and talking about it on stage and then help helped me whilst I integrated it into newgem as a generator and then using it internally itself. For a day and a half we hung out in the hotel foyer. He’s so wonderful.

Cucumber makes me so happy.

Summary

Use newgem. Write gems.

Use mrbones. Write gems.

Use sow. Write gems.

Use gemhub. Write gems.

And write cucumber feature descriptions first. Then unit tests. Then code. Then release. Then profit.

[1] All claims of perfection are for the express purpose of making you try the product enough to use it, share it with your friends, and wrap it up and give it as a gift to family on Xmas day. Gift boxes are available upon request.

Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle)

Already know C extensions in RubyGems? Cool – then just run the following cmds and see what can be generated for you; plus check out the TextMate bundle at the bottom.

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test
rake test

For everyone else…

Its 15000km from Brisbane AU to Prague CZ where Euruko2008 – the European Ruby Conf – was held. I came ready to talk, to met lots of cool multi-lingual Rubyists, and to learn. Ooh, I learnt something alright.

Tim Becker was introducing Native C Extensions for Ruby, and fortunately he said “now, everyone, follow along with this example”. I’d never done native C extensions, but I’d received lots of requests from RubyGem developers on how to do it. I had no idea.

So I was typing in everything Tim told me to type in. When Tim changed slides too quickly, I may have yelled at him to slow down. Perhaps I was the only one doing his tutorial out of 300 people, but I didn’t care. This was gold.

After he finished his session, I dragged him off into the corridor with Jonas Pfenniger (zimbatm), and the three of us mapped out a generic layout for how native C extensions work within RubyGems. I didn’t know any of this, but Tim and Jonas did, and we probably looked silly sitting at a small table in the middle of the narrow corridor.

But at the end, we had a working RubyGem with native C extensions that were built: when the tests were executed via rake, and when the gem was installed. The next day I figured out how to get the C extension built via autotest.

Thanks to Tim and Jonas I was able to then write a extconf generator for RubyGems so that its super-super easy to get started writing native C extensions within RubyGems.

Tutorial

This tutorial is for *nix, as I’m still investigating win32 extensions, and jruby + .net/ironruby extensions. So when I figure that out – hopefully with the help of other people currently at RubyFools conferences, I’ll get back to you.

The code comes from the Pickaxe book – p264, and we’ll insert it into a new RubyGem using newgem (version 0.20.1+):

sudo gem install newgem
newgem pickaxe
cd pickaxe
script/generate extconf my_test

Create a test for a class MyTest that doesn’t exist yet:

# test/test_my_test_extn.rb
require "test/unit"
require 'pickaxe'

class TestMyTestExtn < Test::Unit::TestCase
  def test_working
    t = MyTest.new
    assert_equal(Object, MyTest.superclass)
    assert_equal(MyTest, t.class)
    t.add(1)
    t.add(2)
    assert_equal([1,2], t.instance_eval("@arr"))
  end
end

Run rake to build the C extension and run the tests. You can also run autottest and it will automatically build the C extension before running the tests.

To create the MyTest class, using the code from p262 of the Pickaxe book:

#include "ruby.h"
static int id_push;

static VALUE t_init(VALUE self)
{
  VALUE arr;
  arr = rb_ary_new();
  rb_iv_set(self, "@arr", arr);
  return self;
}

static VALUE t_add(VALUE self, VALUE obj)
{
  VALUE arr;
  arr = rb_iv_get(self, "@arr");
  rb_funcall(arr, id_push, 1, obj);
  return arr;
}

VALUE cTest;
void Init_my_test() {
  cTest = rb_define_class("MyTest", rb_cObject);
  rb_define_method(cTest, "initialize", t_init, 0);
  rb_define_method(cTest, "add", t_add, 1);
  id_push = rb_intern("push");
}

To lib/pickaxe.rb:

require "my_test.so"
# or require 'my_test' if its unique

The last line will import the generated shared library. If the RubyGem is tested or installed on Windows, then the .dll file will be automatically loaded instead. The “.so” notation is merely a placeholder to explicitly specify the shared C-extension, rather than any Ruby library of the same name.

Now run tests (rake), the C extension will be rebuilt and the tests will pass.

Build and install RubyGem

rake manifest:refresh
rake install_gem
irb -rubygems -rpickaxe
> a = MyTest.new
> a.add 3

You have successfully created a C-extension within RubyGems, using TDD.

TextMate bundle for Ruby C extensions

I’ve started a TextMate bundle to give syntax highlighting + some simple snippets for developing the C files for Ruby extensions.

To install:

cd ~/Library/Application Support/TextMate/Bundles
git clone git://github.com/drnic/ruby-c-extensions-tmbundle.git "Ruby C Extensions.tmbundle"

or 

wget http://github.com/drnic/ruby-c-extensions-tmbundle/tarball/master
tar xfv drnic-ruby-c-extensions-tmbundle-master.tar.gz
mv drnic-ruby-c-extensions-tmbundle-master "Ruby C Extensions.tmbundle"

Then restart TextMate or “Reload Bundles”.

You can clone/fork the source via http://github.com/drnic/ruby-c-extensions-tmbundle/tree/master

RubiGen video from RejectConf Berlin

Go straight to video below

I took a video recorder to the RejectConf, like I did in Portland. Unfortunately, there were two reasons I didn’t record any of them.

Firstly, there didn’t seem to be any obvious place to position the camera.

Secondly, it was deemed critical that everyone does their talks in the dark. The conference isn’t run in the dark, local Ruby groups don’t run meetings in the dark but consecutive RejectConfs have been run by adminstrators with a dark fetish. Great for drinking beer and heckling presenters. Bad for video recording.

The makers of the JVC HDD camcorder – a nifty device with a 20G HDD in it – don’t make it possible to record in the dark. Not because it doesn’t have a “night time mode” – apparently it does – but when you are already in the dark, you can’t figure out how to turn it on.

Ok, fine, if I’d made an effort I could have figured it out. So, let’s use excuse #1 as the reason for not recording the presentations.

Its a valid excuse, the Pirate Cove was chock-a-block full of people. The local Berlin Ruby Group did an awesome job of finding a great “underground”-esque venue.

Fortunately a fellow Australian – Marcus Crafter – had a front row position, and a MacBook Pro. With said device, he captured my talk on RubiGen (and John Barton’s).

RejectConf video of RubiGen

In 5 minutes, I make a Merb generator, using RubiGen and NewGem. Nifty stuff indeed.

Recorded and published by Marcus Crafter

There were lots of other awesome presentations (that is perhaps a dubious inference that mine was awesome), but it was dark, I had beer in both hands, and I was too busy yelling “AUSTRALIA!!!” to write notes.

What a great night :)

NewGem Generator – now with script/generate

The New Gem Generator (0.13.0)’s newgem command now behaves like the rails command thanks to RubiGen – a new project that is an extraction of the rails_generator.

Developing a RubyGem? You can now have a script/generate command, just like Rails. Other projects, say like Merb, can do this too.

Awesomeness with a Gold Star. (disclosure: Dr Nic awarded this Gold Star to himself)

Warning

This article can get confusing.

Especially around the heading “Generating Generators”.

Plus, this article talks about two projects – an update to New Gem Generator (0.13.0) and the new RubiGen project.

So, give yourself time to digest it all, and then perhaps come back and read it again.

And then perhaps re-write the article for me.

“A generator that can provide generators to your projects, so that you can write generators for your other projects.”

Re-write it – I dare you.

Background

One of the killer features of Rails is the Rails Generator.

It does three things:

  • generates an entire scaffold for your application, thus sharing with you its conventions (over configuration) methodology; and
  • it then allows you to generate more stuff, like models, controllers, plugins etc. Things that are relevant to a Rails app. Finally,
  • it allows you to write your own generators for Rails apps.

Now you can do all this with New Gem.

Killer Feature #1 – upgrade path

Never before have you been able to run ‘newgem’ on top of an existing RubyGem. Why? It used to blow away the entire folder, and then start writing new files… not very friendly.

Now, just like the rails command, you can go into your RubyGem development directory, and run newgem . and you will be asked which files to override.

NOTE: Copy your Rakefile to Rakefile.old, and after running newgem . copy the configuration information into the new file config/hoe.rb.

If you’re using newgem already, I dare you to upgrade as above. (Ok, perhaps make a backup copy of your work first…)

Killer Feature #2 – script/generate

Once you’ve upgraded, or create a new RubyGem (using newgem gemname), you’ll now have two scripts: script/generate and script/destroy, just like Rails has. (and matching generate.cmd and destroy.cmd for Windows users)

Try them out:

script/generate
...
Installed Generators
  Rubygems: application_generator, component_generator, executable, install_jruby, install_rspec, install_website
  Builtin: test_unit

Oh yeah, I’ve had some fun extracting things into generators.

  • Install the dubious-looking NewGem website – script/generate install_website.
  • Install RSpec support – script/generate install_rspec.
  • Make the RubyGem a JRuby gem – script/generate install_jruby (the generated gem will have -jruby.gem in its name)
  • Create an executable Ruby app – script/generate executable appname

These generators are also reused via the newgem commands various options (Run newgem to see them.)

Killer Feature #3 – generate generators

I’ll post more about RubiGen later, but you can create a new rails-like command-line app that generates a whole stack load of directories and files using script/generate application_generator appname.

There is a large USAGE rundown if you run script/generate application_generator.

Want to create your own generators for developing RubyGems? (similar to creating a generator for a Rails app, but for RubyGems)

script/generate component_generator foobar rubygems
    create  rubygems_generators/foobar/templates
    exists  test
    create  rubygems_generators/foobar/foobar_generator.rb
    create  test/test_foobar_generator.rb
    create  test/test_generator_helper.rb
    create  rubygems_generators/foobar/USAGE
    readme  readme

Firstly, note test/test_generatorname_generator.rb – that’s right, you get a test stub for your new generator. Start there, write tests, then write your generator. There’s inline help for useful assertions.

Secondly, note rubygems_generators folder. This folder is the “scope” of the generator. As it starts with “rubygems” the generator will only be available when you are developing rubygems. It will not show up in Rails, nor Merb or Camping or any other place that may support RubiGen one day.

Similarly, if you want to write a Rails generator using the component_generator then specify the scope as rails.

script/generate component_generator booya rails
     create  rails_generators/booya/templates
     exists  test
     create  rails_generators/booya/booya_generator.rb
     create  test/test_booya_generator.rb
  identical  test/test_generator_helper.rb
     create  rails_generators/booya/USAGE
     readme  readme

In Edge Rails (and any Rails version after 1.2.3), the script/generate command in Rails will search all RubyGems for /rails_generator/* folders in addition to the existing search paths (ticket).

Thanks goes to Rails Generator

I’ve long been in love with the Rails Generator for what it does, the beautiful syntax for specifying a generator, etc. I don’t know who wrote what bit, e.g. if DHH wrote all of it, or others wrote nice chunks, but its awesome. Thanks DHH and co. Thanks to Jeremy Kemper

RubiGen is 95% Rails Generator code, with extensions to support scoping. Setting Rails Generator free of its Rails constraints is a tribute to it.

Now RubyGems can use generators, and any other frameworks can integrate generators and their developers can write and distribute additional generators for those frameworks.

I award myself 2 gold stars, and a scratch-n-sniff

Smart people doing smart things in Netherlands – RubyEnRails 2007

Summary: click to select

rubyenrails2007-me-presenting-by-fngtps.jpg

Photo by Thijs van der Vossen,
Fingertips

For one million people in Amsterdam, time stood still as a parade of Rails gurus in Netherlands/ Belgium talked up a storm at RubyEnRails 2007. Only 240 were allow inside to the free, one day conference, so riots ensued at the doorsteps of the building. Police were called in. Helicopters. Tanks.

Or so I’m told. I was too busy inside meeting many very clever developers and sharp business people. The buzz of excitement for Rails in the foyer during the breaks was tremendous. The conference organisers recorded the audio of all presentations which will be great to listen to.

I was flown in my private jet, helicoptered to the conference building and shepherded in by 24 Dutch Special Forces men, all of equal height and build, identical in every fashion. Before my keynote, I was interviewed by all the major TV networks. I have no idea what drives me to make this all up.

But I was given a free RubyEnRails conference t-shirt. I was also able to give a couple of presentations:

Keynote

It was a wonderful opportunity to start the day, and I shared “What’s cool with Rails?” focusing on some hot potatoes like scaling and deployment, as well as RESTfulness. Slides here.

Ultimately the basic theme of the keynote was “Its all just text”, and choosing between Rails, Merb, Camping, Mongrel handlers, becomes a design/architectural decision to be made for each portion of each application you build.

I included a live demo of the Magic Model Generator, and encouraged people to write RubyGems instead of Rails plugins, as they provide better support for: dependencies, versioning, and reuse outside of Rails.

DIY syntax

The ability to extend my programming language tickles me pink. Often you write a block of code and you just think “That should be prettier and simpler”.

With Ruby meta-programming, blocks, method_missing, const_missing and optional parentheses you can craft nearly any syntactic sugar you like to replace lengthy, complicated code.

Slides here.

Dr Nic Academy

I also took the opportunity to introduce Dr Nic Academy – training courses in Ruby on Rails by me. The first course will be 7th and 8th of July in Amsterdam. If you are thinking of attending, hold off buying tickets!! The new website and a special bonus discount price will be coming soon. I’d hate you to miss out.