At this point in time there’s no facility in Rails to talk to more than one database at a time.

Alex Payne

I possibly have such a facility. Perhaps it will help, and I will get some DHH-love and perhaps a free Twitter account for my troubles. Or perhaps a t-shirt.

As a bonus, the solution even includes decent Ruby-fu syntax. So, if you’re just here for the view:

class PeopleController < ApplicationController
  def index
    @people = conn::Person.find(:all)
  end
end

That code just there solves all our problems. It will invoke Person.find(:all) on a random database connection to (assumably) a clone database. Awesomeness I think. I hope it helps Twitter and all the Twit-sers (or whatever you call a user of Twitter).

This solution comes from the magic_multi_connections gem.

What is going on here?

I think a tutorial is the best way to demonstrate what is happening here. So, let's create a rails app and mix in the magic_multi_connections gem.

First, get the gem. Second, create a rails app:

$ sudo gem install magic_multi_connections
$ rails multi -d sqlite3

Now edit the config/database.yml file to create some more databases:

development:
  adapter: sqlite3
  database: db/development.sqlite3
  timeout: 5000

development_clone1:
  adapter: sqlite3
  database: db/development_clone1.sqlite3
  timeout: 5000

development_clone2:
  adapter: sqlite3
  database: db/development_clone2.sqlite3
  timeout: 5000

But please pretend these are uber-MySQL clusters or whatever.

Think of :development as the read-write connection, and the :development_cloneN connections are for read-only access.

At the bottom of your environment.rb file, add the following:

require 'magic_multi_connections'
connection_names = ActiveRecord::Base.configurations.keys.select do |name|
  name =~ /^#{ENV['RAILS_ENV']}_clone/
end
@@connection_pool = connection_names.map do |connection_name|
  Object.class_eval <<-EOS
    module #{connection_name.camelize}
      establish_connection :#{connection_name}
    end
  EOS
  connection_name.camelize.constantize
end

Let's test what this gives us in the console:

$ ruby script/console
>> @@connection_pool
=> [DevelopmentClone1, DevelopmentClone2]
>> DevelopmentClone1.class
=> Module
>> DevelopmentClone1.connection_spec
=> :development_clone1

Our new modules will act as connections. One module per connection. The code above gives them names to match the connection names, but its really irrelevant what they are called, thanks to the mysterious conn method.

So, go create some models and some data. I'll use Person as the class here.

To setup the schemas in our clone databases, we'll use rake db:migrate. To do this:

$ cp config/environments/development.rb config/environments/development_clone1.rb
$ cp config/environments/development.rb config/environments/development_clone2.rb
$ rake db:migrate RAILS_ENV=development
$ rake db:migrate RAILS_ENV=development_clone1
$ rake db:migrate RAILS_ENV=development_clone2

To differentiate the databases in our example, assume there are two Person records in the :development database, and none in the two clones. Of course, in real-life, they are clones. You'd have a replicate mechanism in there somewhere.

Now, we can access our normal Rails modules through our connection modules. Magically of course.

>> ActiveRecord::Base.active_connections.keys
=> []
>> Person.count
=> 2
>> ActiveRecord::Base.active_connections.keys
=> ["ActiveRecord::Base"]
>> DevelopmentClone1::Person.count
=> 0
>> ActiveRecord::Base.active_connections.keys
=> ["ActiveRecord::Base", "DevelopmentClone1::Person"]

Wowzers. Person and DevelopmentClone1::Person classes?

But note - Person.count => 2 and DevelopmentClone1::Person.count => 0 - they are accessing different databases. The same class definition Person is being used for multiple database connections. We never defined more Person classes. Just the standard default one in app/models/person.rb.

The active_connections result shows that DevelopmentClone1::Person has its own connection. Yet you never had to manually call DevelopmentClone1::Person.establish_connection :development_clone1 - it was called automatically when the class is created.

Of course, DevelopmentClone2::Person is automatically connected to :development_clone2, and so on.

Behind the scenes

Let's look at our generated classes:

$ ruby script/console
>> DevelopmentClone1::Person
=> DevelopmentClone1::Person
>> Person
=> Person
>> DevelopmentClone1::Person.superclass
=> Person

That is, there is a DevelopmentClone1::Person class, automagically generated for you, which is a subclass of Person, so it has all its behaviour etc.

Dynamic connection pools within Rails controllers

The magic of the conn method will now be revealed:

$ ruby script/console
>> def conn
>>   @@connection_pool[rand(@@connection_pool.size)]
>> end
>> conn::Person.name
=> "DevelopmentClone2::Person"
>> conn::Person.name
=> "DevelopmentClone1::Person"

The conn method randomly returns one of the connection modules. Subsequently, conn::Person returns a Person class that is connected to a random clone database. Booya. Free Twitter swag coming my way.

Place the conn method in the ApplicationController class, and you can get dynamic connection pooling within Rails actions as needed, as in the following example (from the top of the article):

class PeopleController < ApplicationController
  def index
    @people = conn::Person.find(:all)
  end
end

Decent Ruby-fu, I think. Certainly better than manually calling establish_connection on model classes before each call (or in a before_filter call, I guess).

This is just a concept

I know this tutorial above works. But that might be the extent of what I know. Let me know if this has any quirks (especially if you solve them), or if this is a stupid idea for implementing connection pooling with nice Ruby syntax.

Hope it helps.

Related posts:

1. Using CoffeeScript in Rails and even on Heroku I’m pretty excited about CoffeeScript as a clean-syntax replacement for...
2. First look at rails 3.0.pre This article is out of date in some aspects....
3. Rails themes can remember things I was getting annoyed at having to remember all the...

Beginner’s Guide to Mapping

Need to invoke a method on each object in an array and return the result?

# call 'to_i' on each item of the list to return the list of numbers
>> list = ['1',  '2', '3']
=> ["1", "2", "3"]
>> list.map {|item| item.to_i}
=> [1, 2, 3]

That is, we called the to_i method on each item of the list, thus converting the 3 strings into 3 integers. Of course, we could have called any method on any set of objects.

Advanced Guide to Symbol.to_proc

After doing that a few times, you start wishing there was simpler syntax. Enter: Symbol.to_proc

>> list.map &:to_i
=> [1, 2, 3]

It looks like magic. Well it certainly doesn’t make any sense. But it works. (The secret: the :to_id symbol is cast into a Proc object by the leading &. That is, it calls the to_proc method on the symbol. Rest of explanation here.)

Magical version of Symbol.to_proc

Quite frankly, that’s still a lot of syntax. Plus, I normally forget to added parentheses around the &:to_i, and then latter I want to invoke another method on the result, so I need to add the parentheses which is a pain… anyway. I thought of something niftier and dare I say, more magical.

How about this syntax:

>> list.to_is
=> [1, 2, 3]

This syntax makes sense – to_is is the plural of to_i – its obvious you want the to_i‘s of your list.

What’s happening here?You pass the plural of the method name to the container and it invokes the singular of the method name upon the children using the map code above. Source is below.

More examples? Want to get the names of all the objects of a list? (assuming that each item in the list has a name method)

>> contacts.map {|contact| contact.name}
=> ['Dr Nic', 'Banjo', ...]
>> contacts.map &:name
=> ['Dr Nic', 'Banjo', ...]
>> contacts.names
=> ['Dr Nic', 'Banjo', ...]

You pick your favourite. I love the last one.

Bonus examples:

>> (1..10).to_a.to_ss
=> ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10"]
>> (1..10).to_a.days
=> [86400, 172800, 259200, 345600, 432000, 518400, 604800, 691200, 777600, 864000]
>> [2,'two', :two].classes
=> [Fixnum, String, Symbol]
>> [2,'two', :two].classes.names
=> ["Fixnum", "String", "Symbol"]
>> [2,'two', :two].classes.names.lengths
=> [6, 6, 6]

So much happy syntax in one place!

UPDATE: After conversation on Ruby Forums, we’ve come up with some new syntax:

>> (1..5).to_a.map_days
=> [86400, 172800, 259200, 345600, 432000]
>> list = [nil, 1, 2, nil, 4]
=> [nil, 1, 2, nil, 4]
>> list.reject_nil?
=> [1, 2, 4]

Neat.

How do I do this at home?

Download and include this mini-library. Source below. Have fun.

module GenericMapToProc
  def self.included(base)
    super

    base.module_eval <<-EOS
      def method_missing(method, *args, &block)
        super
      rescue NoMethodError
        error = $!
        begin
          re = /(map|collect|select|each|reject)_([\\\\w\\\\_]+\\\\??)/
          if (match = method.to_s.match(re))
            iterator, callmethod = match[1..2]
            return self.send(iterator) {|item| item.send callmethod}
          end
          return self.map {|item| item.send method.to_s.singularize.to_sym}
        rescue NoMethodError
          nil
        end
        raise error
      end
    EOS
  end
end

unless String.instance_methods.include? "singularize"
  String.module_eval <<- EOS
    def singularize
      self.gsub(/e?s\Z/,'')
    end
  EOS
end

# Add this to the Array class
Array.send :include, GenericMapToProc

Related posts:

1. map_by_method now works with ActiveRecord associations I was always annoyed that map_by_method was broken for ActiveRecord...
2. Magic Wiggly Lines => GuessMethod, by Chris Shea If you ever make time to code just for pleasure,...
3. Magic Multi-Connections: A “facility in Rails to talk to more than one database at a time” At this point in time there’s no facility in Rails...

Here’s an example definition of some terrain:

  class Desert < Terrain
    title           "Desert"
    letter          "d"
    graphic         "desert"
    movement_cost   1
    defense_bonus   10
    food            0
    shield          1
    trade           0
  end
  class Oasis < Desert
    title           "Oasis"
    special_code    1
    food            3
    shield          1
    trade           0
    graphic         "oasis"
  end

An Oasis is a special version of the Desert terrain, so Ruby subclasses offer a compatible relationship.

This is seriously cool. All in Ruby.

Why write a DSL for something trite like game rules?

The descriptions of terrain (plains, hills, ocean), the improvements (roads, irrigation, mines), etc. all need defining somewhere. Here are the standard options:

1. Rows in a database, imported into ActiveRecords at startup
2. Files that are loaded at startup, parsed and converted into an internal data model
3. Ruby DSL

The sort of configuration we're dealing with is static: its a part of the game design to a certain extent. Yes, I could put the configuration in a database and access/modify it via an admin console, though version control becomes an additional administrative hassle.

Storing game rule configuration in external files makes version control trivial, though there is a one-time cost of implementing special purpose syntax, parsers, internal game data models and testing.

A Ruby DSL is exactly the same as implementing option 2 - the configuration is stored in files, though as the syntax is pure Ruby, there is no need to implement a parser and internal data model.

There was a one time cost for supporting the syntax. But why the lucky stiff paid this cost for me with his traits definition for slaying dragons [1]. So I was free and clear of any actual effort.

"Yes, smart arse, but you still had to write up all those Ruby classes"

An irrelevant argument, you'd have to enter the configuration for any of the 3 options, but...

I wrote a generator to build the Ruby class definitions for me from the Freeciv's GPL rule sets. Sweet.

[1] The definition of the Terrain class, using the traits mechanism, is simple:

  class Terrain
    traits :title,
      :letter,
      :graphic,
      :movement_cost,
      :defense_bonus,
      :food,
      :shield,
      :trade,
      :special_code
  end

Related posts:

1. Magic Multi-Connections: A “facility in Rails to talk to more than one database at a time” At this point in time there’s no facility in Rails...
2. Coming home to Brisbane I left Australia on the 3rd of July last year...
3. Extending _why’s Creature class Many Rubist’s first explanation of metaprogramming is by why the...

The effect

See if your unsuspecting friend can figure out what this code will do. If they can, then say, “That’s because you are a domain expert in English. Good for you. Now use Ruby you Java boy.”

>> I.say "I love the Ruby language"
You said, 'I love the Ruby language'

Cool, yes? Yes.

How this trick works

To set this up for your unsuspecting Ruby noob, type the following into your console/irb:

>> class I; def self.say(text); puts "You said, '#{text}'"; end; end
=> nil

Now you have a constant I upon which you can call class methods.

In long hand, this is:

class I
  def self.say(text)
    puts "You said, '#{text}'"
  end
end

Create other pronoun classes, such as You, Everyone, etc and give them methods representing verbs, such as say, said, want_to.

For example,

class You
  def self.cannot(action, target)
    puts "Didn't your mother tell you never say never? Of course you can #{action} #{target}"
  end
end

Which delights us with:

>> You.cannot :learn, "Ruby"
Didn't your mother tell you never say never? Of course you can learn Ruby

“I still don’t know what a DSL is”

That’s unfortunate for you.

Related posts:

1. Validate and Save your Ruby in TextMate – with secret Rubinus superpowers In some TextMate bundles, if you save a file it...
2. Future proofing your Ruby code. Ruby 1.9.1 is coming. Bugger. I’m a Ruby monogamist. I use the Ruby...
3. Unit Testing iPhone apps with Ruby: rbiphonetest Everything to love about Ruby: the concise, powerful language;...

But cattr_accessor doesn’t work the way you (read, “me”) thought at first glance when you use subclasses. I thought if I declared a class accessor in the superclass, then I would have independent class attributes for all my subclasses. Apparently not…

>> class Parent; cattr_accessor :val; end
=> [:val]
>> class Child1 < Parent; end
=> nil
>> class Child2 < Parent; end
=> nil
>> Child1.val = 4
=> 4
>> Child2.val
=> 4
>> Child2.val = 5
=> 5
>> Child1.val
=> 5

Child1.val and Child2.val seem to be the same value. Not very independent at all. Internally, the classes share a common class attribute. This is useful in certain circumstances, but not what I was looking for.

Instead, I found class_inheritable_accessor.

>> class Parent; class_inheritable_accessor :val; end
=> [:val]
>> class Child1 < Parent; end
=> nil
>> class Child2 < Parent; end
=> nil
>> Child1.val = 4
=> 4
>> Child2.val = 4
=> 4
>> Child2.val = 5
=> 5
>> Child1.val
=> 4

Lovely. Each subclass will have an independent value once you’ve assigned it a value explicitly, else it will pick up the value from its superclass.

UPDATE: class_inheritable_accessor and co. actually clone the superclass’s inherited attributes, rather than just referencing them.

Related posts:

1. Magic Multi-Connections: A “facility in Rails to talk to more than one database at a time” At this point in time there’s no facility in Rails...
2. Turn-based game DSL Late in the night, whilst the baby feeds, I continue...
3. Extending _why’s Creature class Many Rubist’s first explanation of metaprogramming is by why the...