Dr Nic

[ANN] Generating new gems for graceful goodliness

Chicken!

I don’t like you [1]. You don’t share code. I know, I know, you’ve had good reasons – you don’t know how to create a Ruby gem, how to upload it to a gem server like RubyForge, and you’re a chicken. Today we’ll remove the first of these minor roadblocks, with a New Gem Generator!

Now you can take any library or Rails plugin or command line application, gemify it, and easily share it with the Ruby world. With gems you get in-built version support (you can specify which version of a gem you want when you use it via the require_gem method), an encapsulated, consistent folder structure for your bin/lib/test folders, and you get cross-platform support for bin apps. Too much niftiness to ignore, really.

The New Gem Generator is like the rails command for rails applications, but it creates the folders and starting files for a new gem. It’s called newgem.

Tutorial

Aim

To convert the Map by Method (previously called Map by Pluralisation) library into a gem.

See original and demo articles. Sex on a stick – soon to be gemified before your very eyes. (Download instructions for the prebuilt gem)

Installation

> gem install newgem

Download from rubyforge if you have firewall problems (as I do at work) and need to get the gem explicitly first. THEN run the above command in the folder you saved the gem.

Create new gem

> newgem map_by_method
creating: map_by_method
creating: map_by_method/CHANGELOG
creating: map_by_method/README
creating: map_by_method/lib
creating: map_by_method/lib/map_by_method
creating: map_by_method/lib/map_by_method.rb
creating: map_by_method/lib/map_by_method/version.rb
creating: map_by_method/Rakefile
creating: map_by_method/test
creating: map_by_method/test/all_tests.rb
creating: map_by_method/test/test_helper.rb
creating: map_by_method/test/map_by_method_test.rb
creating: map_by_method/examples
creating: map_by_method/bin

Copy in the library

The generated lib/map_by_method.rb file looks like:

Dir['map_by_method/**/*.rb'].sort.each { |lib| require lib }

This will automatically include (require), in alphabetical order, the files in the
lib/map_by_method folder. If you need the files required in a specific order, then do it explicitly here, for example:

require 'foo'
require 'bar'
require 'tar'

or

%w(foo bar tar).each {|lib| require lib}

For this gem there won’t be any additional library files, so we’ll just copy in the following code into the lib/map_by_method.rb file and remove the library loading code.

module MapByMethod
  def self.included(base)
    super

    base.module_eval <<-EOS
      def method_missing(method, *args, &block)
        super
      rescue NoMethodError
        error = $!
        begin
          re = /(map|collect|select|each|reject)_([\\w\\_]+\\??)/
          if (match = method.to_s.match(re))
            iterator, callmethod = match[1..2]
            return self.send(iterator) {|item| item.send callmethod}
          end
          return self.map {|item| item.send method.to_s.singularize.to_sym}
        rescue NoMethodError
          nil
        end
        raise error
      end
    EOS
  end
end

unless String.instance_methods.include? "singularize"
  class String
    def singularize
      self.gsub(/e?s\Z/,'')
    end
  end
end

Array.send :include, MapByMethod

Package your gem into a .gem file

From the root folder of your gem run rake package:

> rake package
(in C:/InstantRails/ruby_apps/map_by_method)
rm -r .config
  Successfully built RubyGem
  Name: map_by_method
  Version: 0.0.1
  File: map_by_method-0.0.1.gem
mv map_by_method-0.0.1.gem pkg/map_by_method-0.0.1.gem

Tada! You are the owner of a gem.

Install your gem onto your machine

Your Ruby (and Rails) applications can only use the gemified libraries once you have installed the packaged gem. This is the same for other people who will use your gem.

Typically you install a gem from a remote gem server such as rubyforge. Today, you will install the gem locally:

> gem install pkg/map_by_method-0.0.1.gem
Attempting local installation of 'pkg/map_by_method-0.0.1.gem'
Successfully installed map_by_method, version 0.0.1
Installing RDoc documentation for map_by_method-0.0.1...

Note that it created and installed RDoc documentation for the library too. Each user automatically has a copy of the generated documentation for your libraries (if you actually added documentation to your code).

Unit testing

Look in the test folder and see that it has created a map_by_method_test.rb unit test file to get you started. Put tests in there. Add more test files. Run rake test from the project’s root folder and watch all your tests succeed or fail. Be good.

Version numbers

Note that the generated gem is map_by_method-0.0.1.gem. The 0.0.1 is the version number of the gem, and you can easily change this as you wish as you gem takes on new features and fixes.

There are two common version number formats:

X.Y.Z – X = major release number, Y = minor release number, Z = patch/bug fix number

or

X.Y.Z.svn = svn is the subversion number at the time the gem was released.

The latter is the default implementation generated by newgem. If you want the simpler version number format (the 1st one), then remove the following line from your Rakefile (around line 13):

REV = File.read(".svn/entries")[/committed-rev="(\d+)"/, 1] rescue nil

Your gem has a prebuilt mechanism for specifying the X.Y.Z portion of the version number.

Go to lib/map_by_method/version.rb

module MapByMethod #:nodoc:
  module VERSION #:nodoc:
    MAJOR = 0
    MINOR = 0
    TINY  = 1

    STRING = [MAJOR, MINOR, TINY].join('.')
  end
end

Change MINOR and TINY to:

    MINOR = 1
    TINY  = 0

And repackage and reinstall your gem:

> rake package
(in C:/InstantRails/ruby_apps/map_by_method)
rm -r .config
  Successfully built RubyGem
  Name: map_by_method
  Version: 0.1.0
  File: map_by_method-0.1.0.gem
mv map_by_method-0.1.0.gem pkg/map_by_method-0.1.0.gem

> gem install pkg/map_by_method-0.1.0.gem
Attempting local installation of 'pkg/map_by_method-0.1.0.gem'
Successfully installed map_by_method, version 0.1.0
Installing RDoc documentation for map_by_method-0.1.0...

And you’re done. Next you would upload your gem to a gem server such as RubyForge, or one your company runs internally to share gems via the “gem install” mechanism.

map_by_method already on RubyForge

If you want the map_by_method gem, its already on RubyForge, so you can remotely install it:

gem install map_by_method

Inspiration

Jay Fields created a great “ruby application setup” script.

[1] That’s not true. I do like you.

Related posts:

  1. My RubyGems development tools and workflow The Open Source Developers Conference (osdc) is a nifty...
  2. newgem 1.0.0 all thanks to Cucumber The New Gem Generator (newgem) was exciting, moderately revolutionary, and...
  3. Writing C extensions in RubyGems using newgem generators (plus a free TextMate bundle) Already know C extensions in RubyGems? Cool – then just...
  4. newjs = newgem for JavaScript projects; free TDD suite Want to start a new JavaScript project for a library...
  5. map_by_method – the final announcement I don’t really talk about my projects after I release...

62 Responses to “[ANN] Generating new gems for graceful goodliness”

  1. Simon Harris says:

    Awesome! I’ve had in mind to write some gems but rails made it too easy to just write plugins so I never got around to it.

  2. Dr Nic says:

    “Rails makes it too easy to …” Let me be the first to infer that Ruby on Rails sounds like some sort of Hard Drug :)

  3. Ryan Davis says:

    You and I have been working on nearly the exact same thing. Check out hoe from seattlerb on rubyforge if you get a chance.

  4. choonkeat says:

    ah.. i’m a happy chicken then. thanks doc!

    (but 3 steps?… hmm)

  5. Dr Nic says:

    @ryan – sorry for the duplication, I’d spotted hoe before but didn’t realised it (recently) had ‘sow’. Perhaps add the newgem templates etc into hoe and then you can do all the maintenance on the project :)

    @choonkeat – I’ve deliberately not counted out the number of steps because its just never few enough for some people! :)

  6. Anonymous says:

    Generating new gems for graceful goodliness…

    “I dont like you. You dont share code. I know, I know, youve had good reasons – you dont know how to create a Ruby gem, how to upload it to a gem server like RubyForge, and youre a chicken. Today well remove the first of these minor roadblocks, with a…

  7. pth says:

    You might want to add a setup.rb to your gem generator. It is my understanding (per Chad) that this is considered good form.

  8. tadatoshi says:

    I have a question.

    I want to execute an application packaged as gem.
    How can I do so?

  9. Anonymous says:

    Dr Nic [ANN] Generating new gems for graceful goodliness…

    Now you can take any library or Rails plugin or command line application, gemify it, and easily share it with the Ruby world. With gems you get in-built version support (you can specify which version of a gem you want when you use it via the require_ge…

  10. Dr Nic says:

    @tadatoshi:

    I have a question.

    I want to execute an application packaged as gem.
    How can I do so?

    I’ll write an article about this later, but in short:

    1. Add a file to the bin directory, say “myscript”
    2. Add myscript in BIN_FILES, eg. BIN_FILES = %w( myscript )
    3. Package and install

    I only figured this out recently (so I could create the newgem application). But its very exciting as your apps are platform-neutral.

  11. Dr Nic says:

    @pth – after talking with you and Chad, v0.3 now includes the setup.rb file (both for itself and for your generated gems)

    Reference: http://i.loveruby.net/en/projects/setup/

  12. Dr Nic says:

    v0.4 – http://rubyforge.org/frs/shownotes.php?release_id=7358

    choonkeat has patched in some command line parameters:

    > newgem
    Take any library or Rails plugin or command line application,
    gemify it, and easily share it with the Ruby world.
    
    Usage: newgem [options] 
    
    Options are:
        -i, --import_path=PATH           Path where your files could be copied from
                                         Default: none
        -v, --version=YOUR_VERSION       Version of the gem you are creating
                                         Default: 0.0.1
        -h, --help                       Show this help message.
    
  13. [...] The eminent Dr. Nic Williams has put together a useful generator that makes it a lot easier to construct your own gems. It’s an ideal script if you’ve never put a gem together before, or if you’re sick of the repetition involved. Nic tells me that this tool might make its way into the core RubyGems distribution in the future, but it’s worth playing with straight away. [...]

  14. Hey, I’m not a chicken! Ok, I guess I am. This looks pretty cool. Nice work.

  15. Anonymous says:

    Dr. Nic : How To Easily Create Gems…

    Dr. Nic shows you how you can take any library or Rails plugin or command line application, gemify it, and easily share it with the Ruby world….

  16. [...] Install the latest version of newgem (minimum v0.5) [...]

  17. soeren says:

    Great idea with perfect timing.

    Why don’t you use the rake/testtask?

    require 'rake/testtask'
    
    Rake::TestTask.new do |t|
     t.libs < < "lib"
     t.test_files = FileList['test/*.rb']
     t.verbose = true
    end
    

    which takes also care of the libs in an easy and understandable way. Your solution didn't find my libs after copying in libs and tests and runnin 'rake test'.

    Newgem came just in time for me since im developing a command line app for some time now. Also SimpleConsole seems promising although I have to figure out how to retain some features not supported directly by the package.

    I thought about gemifying it, but hesitatet. With newgem, gemifying it
    is a breeze.

  18. Dr Nic says:

    @soeren – So, rake testing is broken? I’ll check it out. Cheers.

  19. Dr Nic says:

    @soeren – SimpleConsole looks very nice for multi-function apps. For complex + simple apps, bundling + deploying via a gem rocks!

  20. Dr Nic says:

    @soeren – ok, updated the generated Rakefile with sexier new test task. Uses pattern “test/**/*_test.rb” now. Thanks.

    Version 0.6 available for download.

  21. [...] Update: this is a gem called map_by_method. [...]

  22. [...] Well, I was going to release a new plugin Saturday morning, but I decided there was an issue to address. Particularly, the skepticism around the plugin_dependencies plugin. Rick Olson did a little piece about plugin_dependencies, and Dr Nic wrote an interesting article a while ago about generating new gems. This all got me thinking (sometimes that’s a bad thing ). [...]

  23. evan says:

    Hey, Nic. map_by_method breaks special rules in the Rails inflector. To whit, in environment.rb:

    Inflector.inflections do |inflect|
    inflect.irregular ‘recipe’, ‘recipes’
    end

    When map_by_method is loaded in ~/.irbrc, the console can’t inflect “recipes” properly. But without the gem, the special rule gets used.

  24. [...] The New Gem Generator rocks. Type newgem <gem_name> and you get all standard scaffolding for a gem. They look just like plugins, or vice versa. And the -b option bangs out new command line apps. Sweet. [...]

  25. Justin says:

    Thanks for the great work Nic!

    Few problems with newgem. The newgem command doesn’t work like the rails command. The rails command is careful not to delete what is in an existing directory path. newgem clobbers over everything. (I was hoping that it would update the directory structure like the rails command.)

    Another problem, and this probably has more to do with Hoe than your code. I have a library with a “core” directory that is clobbered with the “clean” task. It did this within the “lib” and “test” subdirectories (both directories had “core” as a subdirectory somewhere along the path).

    Feel free to email me if you like.

  26. Dr Nic says:

    @justin – yeah, the original code I worked with starts with the following equivalent rm -rf on the folder… soooooo…. that will probably explain your symptoms :)

    In a future version I’ll leverage the railties generation system to support the overwrite? [Ynaq] etc. Or if you are feeling keen, perhaps you’d like to have a go at doing it?

  27. [...] Update: Now available via gems, thanks to Dr. Nic’s newgem magic: [...]

  28. Tatsuya says:

    I hava a question, too.
    I’d like to create an extension package including extconf.rb.
    How can I do it ?

  29. herry says:

    Dr. Nic, was just wondering how do you use newgem for subpackage libraries e.g. net/http

  30. Glenn Rempe says:

    Hi Dr. Nic. Great gem. Here are a couple of things that look to me from the Hoe docs need to be in the rakefile that is generated. What do you think? Otherwise it seems to me the vars that you have in the file (e.g. RDOC_OPTS) never get used. The name and version options are required according to the Hoe documentation ( http://seattlerb.rubyforge.org/hoe/ ).

    p.name = NAME
    p.version = VERS
    p.spec_extras = {
    :extra_rdoc_files => ["README.txt", "History.txt", "CHANGELOG.txt"],
    :rdoc_options => RDOC_OPTS,
    :autorequire => GEM_NAME
    }

    Also, I can’t seem to make the Hoe p.extra_deps work if I try to specify a version number for a required Gem. Do you have a working example you can share?

    Thanks for getting me over the hurdle of my first Gem.

    Glenn

  31. nicolas says:

    Lord Nic, thank you for all this goodness

    Nicolas

  32. Glenn Gillen says:

    I keep getting an “undefined method ‘exitstatus’ for nil:NilClass” when doing my rake package.

    Any ideas?

  33. Dr Nic says:

    Yeah, Windows users need to install tar + zip as required by hoe; though check out hoe docco if you don’t want to generate tar + zip files.

  34. Orion says:

    Hi Dr. Nic, It seems that when I package up my gem using your system, it only includes the version.rb from the lib directory and none of my other files

  35. Dr Nic says:

    The Hoe library is now included in newgem (see here). Now you need to update your Manifest.txt file with the full path of each file to be included in the package.

  36. Robert Berger says:

    I am using qtRuby in my project and I create GUI files using the QT Designer which produces these xml files. There is a “compiler” called rbuic that takes those files and produces ruby files.

    Any suggestions on how could I fit that into the tree created by newgem?

    Before using it in a gem, I created my own makefile that looks like this:

    RBUIC=rbuic

    RBUICIMPLS=main_form.rb \
    gps_options_form.rb \
    ping_options_form.rb \
    iperf_options_form.rb \
    sniff_radio_options_form.rb \
    data_radio_options_form.rb \
    not_implemented_form.rb \
    error_form.rb \
    dhclient_form.rb \
    qmake_image_collection.rb

    all: $(RBUICIMPLS)

    main_form.rb: main_form.ui
    $(RBUIC) -x $

  37. Robert Berger says:

    Looks like my last post got truncated.

    The main thrust of the question is where to put the .ui xml files that are then translated to ruby files using hte rbuic “compiler” that define the classes “required” by my code in the lib/ directory of the gem?

    Should I put the ui files in ext/ Should I cause the rbuic commands be fired off by the main rakefile or create an extconf.rb that creates a makefile and runs it in ext/ If so where should it put the resulting libraries?

    Or is there some other proper way to do this?

    Thanks!
    Rob

  38. Dr Nic says:

    I highly recommend the RubyGems-Developer forum for such questions (that is, I don’t know the answer to your question :)

    Forum details

  39. Jay says:

    Hi,

    I can’t for the life of me figure out how to package a generator as a gem using your plugin.. I created a “modified_scaffold_generator” in my ~/.rails/generators/ directory, it works perfectly.. now I want to package that as a gem.. I copied everything to the lib directory created by your gem, but it doesn’t create anything when I run my modified scaffold generator.. so any ideas how to do this?

    Thanks

  40. Dr Nic says:

    @jay – the latest update includes the hoe gem; subsequently it enforces the use of Manifest.txt. You need to list each file you want to include in the gem in the Manifest.

  41. tar.zip says:

    how to install “tar + zip” in windows?
    any gems ?

  42. iobass says:

    Just installed newgem (newgem-0.9.4) on a Windows system and I get the following error.

    c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require’: no such file to load — active_support (LoadError)
    from c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require’
    from c:/ruby/lib/ruby/gems/1.8/gems/newgem-0.9.4/bin/newgem:10
    from c:/ruby/bin/newgem:18:in `load’
    from c:/ruby/bin/newgem:18

    Is a dependency missing?

  43. @Dr Nic [via] – you mentioned making your newgem command runnable by adding it to a constant called BIN_FILES somewhere. Where does that live? Could you give a few more details on how that works?

    Thanks!

  44. [...] You may also want to look at the prolific Dr. Nic’s newgem. I like sow because it generates the smallest number of files necessary to make a gem. newgem is more powerful and provides a lot more functionality. Try both and see which one you like. [...]

  45. Kalivo.com says:

    Generating a Gem…

    Most of the following is taken from Dr. Nic…

  46. diwyata says:

    I got error when used ‘rake package ‘ command

    c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require’: no such file to load — active_support (LoadError)
    from c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require’
    from c:/ruby/lib/ruby/gems/1.8/gems/newgem-0.9.4/bin/newgem:10
    from c:/ruby/bin/newgem:18:in `load’
    from c:/ruby/bin/newgem:18

  47. Dr Nic says:

    @diwyata – Looks like you’re running an old version of newgem (0.9.4).

    Run “gem install newgem” to get 0.16.1 and it should install the correct dependencies, and you should be off and running. Bug averted.