Dr Nic

Showing off data on a timeline

I’m still trying to justify my effort writing the MagicCGI code. It let you get an XML or JSON feed for any database, with some basic conditionals, limits etc. The existing demo is for my blog database.

I think this one is kinda cool – showing off all your blog posts/articles on a timeline:

Can’t see the snazzy timeline above?

To get the timeline working with the schema output from the Magic CGI, I needed to write my own Timeline EventSource. Currently I don’t do anything fancy with the generated bubbles – I just use the defaults.

The MagicCGI query gets all wp_posts (WordPress schema) rows, where “post_status=publish” and only returns fields that are relevant (notably ignores the large post_content field holding the blog content):


The other cool thing I did here was to deploy it all with Capistrano (an html page + javascript libs). It even deploys/manages a copy of the Timeline trunk onto the server. This is the first time I’ve deployed a non-Ruby/Rails app using Capistrano, and once I got it set up it becomes much easier to manage than using an FTP app, etc.

How easy? cap1 update (note the dubious use of capistrano 1… I still… haven’t… converted… to 2.0…)

I’m not confident enough that my solution is sexy enough to outline in detail, so if you’re interested in deploying Javascript apps etc with Capistrano, just checkout the code (below) and look at the config/deploy.rb script. (note that I’ve disabled the deprec require statement as it assumes I’m wanted to run some mongrels etc, but deprec is very handy for setting up ssh and other fun stuff at the start).

So, no details here, just a fun example.

If the Timeline tickles your fancy, their website has lots of tutorials, and/or check out my html/javascript code.

svn co http://drnicwilliams.com/svn/blog_timeline/trunk blog_timeline

If the MagicCGI tickles your fancy, its also only on svn at the moment, though its docco should be pretty good. I think.

svn co http://rubyforge.org/var/svn/magicmodels/magic_cgi/trunk magic_cgi

MagicCGI shows OpenID user count

OpenID count

In the last 20 days, 43 people have used OpenID to leave comments. That’s very cool.

Corollary: add OpenID login to your blog.

Even cooler – Dynamic counter

The screen shot comes from the Comments form.

If you look at the comments form in a week, month, year, the counter above have be changed from its original value 43.

No fancy WordPress plugins (I don’t do PHP)

No Apache tricks. (I don’t know any Apache tricks)

Instead with Javascript/HTML attached to JSON attached to a RubyCGI script attached to my WordPress database via ActiveRecords and some magic.

I call it MagicCGI. I also call it Frigging Scary.

Try the following:

$ curl -v "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_openid_identities&action=count&format=json"
< Content-Type: txt/json

Or some XML?

$ curl -v "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_openid_identities&action=count&format=xml"
< Content-Type: txt/xml
<?xml version="1.0" encoding="UTF-8"?>
  <count type="integer">43</count>

Raw data?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_posts&field=post_title&limit=5&order=post_title"
[{"attributes": {"post_title": "OpenID count"}},
 {"attributes": {"post_title": "MagicCGI shows OpenID user count"}},
 {"attributes": {"post_title": "map_by_method now works with ActiveRecord associations"}},
 {"attributes": {"post_title": "Feedburner"}},
 {"attributes": {"post_title": "One year on the InterTubes"}}

Getting kinda scary now, I think.

Ooh, just how much magic?

Want a list of available tables to play with?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?meta=tables"
[{table_name: 'wp_users'},...]
  • add &format=xml to XML output; JSON is default
  • add &meta=columns to include the column schema definitions
  • add &table_name=wp_posts for each table you want (instead of all the tables)

E.g. to see the columns for wp_posts and no other table, in XML:

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?meta=tables&meta=columns&table_name=wp_posts&format=xml"

A list of all urls and internal user_ids for users/OpenID users/registered commenters?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_users&field=user_url&field=id"
[{"attributes": {"id": "1", "user_url": "http://drnicwilliams.com"}}, ...

So, now we know Dr Nic = user id 1.

Oh oh oh, how about a list of comments for a specific user?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_comments&user_id=1"
...comments by Dr Nic...
$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_comments&user_id=1&action=count"

Dr Nic’s commented in his own blog 232 times? Out of how many comments?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_comments&action=count"

Where are the user emails? Where are their passwords?

Hidden! See below.


JSON URLs support callback=someCallbackMethod and/or variable=someLocalVariable.

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_comments&action=count&callback=someMethod"

What does wp_drnicwilliams.cgi look like?

Something a little like this…

require 'magic_cgi'         # loads the render magic, model magic, and meta-model magic
include Render

require 'magic_cgi/config/wordpress'  # connect to DB using WordPress installation (wp-config.php)
MagicCGI::Config::Wordpress.establish_connection "/path/to/drnicwilliams/web/public"

# The following is defaulted for WordPress connections:
MagicCGI::Config.hidden_tables |= %w[wp_openid_nonces wp_openid_associations wp_usermeta wp_tla_rss_map wp_tla_data]
MagicCGI::Config.hidden_columns['wp_users'] = %w[user_email user_pass user_activation_key]
MagicCGI::Config.hidden_columns['wp_openid_identities'] = %w[hash]
MagicCGI::Config.hidden_columns['wp_comments'] = %w[comment_author_email comment_author_IP]
MagicCGI::Config.hidden_columns['wp_posts'] = %w[post_password]

render do |params|
  data = DbTable.from_params(params)
  data ||= begin
    table_name = params['table'].first || 'wp_users'
    model_name = ActiveRecord::Base.class_name(table_name)
    klass = MagicCGI::MagicModel.create_class(model_name, ActiveRecord::Base)

Can I write my own CGI scripts for my own DBs?

Sure. Its a library called MagicCGI.

What is MagicCGI?

Coming soon.

map_by_method now works with ActiveRecord associations

I was always annoyed that map_by_method was broken for ActiveRecord has_many associations. 6 mths later I finally fixed it.

That’s the magic of Open Source Software. [/end sarcasm]

So now, the following example works like it should:

$ gem install map_by_method
$ console
> require 'map_by_method'  # stick this in your environment.rb for Rails
> user = User.find_by_name "Dr Nic"
> user.companies.map_by_name
=> ['Dr Nic Academy', 'Dr Nic Institute of Being Silly']
> user.companies.map_by_id_and_name
=> [[1, 'Dr Nic Academy'], [9, 'Dr Nic Institute of Being Silly']]

Recap: why use map_by_method?

Try the following example:

> user.companies.map_by_employees.flatten
=> list of all employees of user


> user.companies.map { |company| company.employees}.flatten
> user.companies.map(&:employees).flatten

Or compare:

> user.companies.map_by_id_and_name
=> [[1, 'Dr Nic Academy'], [9, 'Dr Nic Institute of Being Silly']]


> user.companies.map { |company| [company.id, company.name]}

That is, it looks and feels just like ActiveRecord’s #find method, with its find_by_first_name_and_last_name magic.


No {, }, |, &, or : required. Just clean method names.

Bonus other gem

In the spirit of ActiveRecord hacks, there is to_activerecord:

$ gem install to_activerecord
$ console
> require 'to_activerecord'  # stick this in your environment.rb for Rails
> [1,2,3].to_user
=> [list of User with id's 1,2,3]

To me, this suffix operator reads cleaner than the traditional:

> User.find([1,2,3])

For example, if you want to perform an operation on the list of Users:

> ids = [1,2,3]
> ids.to_user.map_by_name
=> ['Dr Nic', 'Banjo', 'Nancy']


> User.find(ids).map_by_name

Magic Wiggly Lines => GuessMethod, by Chris Shea

If you ever make time to code just for pleasure, then method_missing and const_missing are just begging for abuse.

Chris Shea has come up with GuessMethod – a very cool hack that now deprecates my concept of Magic Wiggly Linesa spell-checker for runtime code.

What’s it do? Cop a squiz at this genius…

$ gem install guessmethod -y
$ irb
> require 'rubygems'
> require 'guessmethod'
> class Object; include GuessMethod; end  # though this could go in the guessmethod.rb file in the gem
> class Product; def name; "Some product"; end; end
> Prodct.nw.nae
attention: replacing non-existant constant Prodct with Product for Object
attention: sending new instead of nw to Product:Class
attention: sending name instead of nae to #<Product:0x144ff10>:Product
=> "Some product"

That’s going straight into my .irbrc file. My bad spelling, coupled with my British/Australian English, will never slow me down again!

UPDATE: actually, it doesn’t like being in the .irbrc file for Rails console; so in the config/environments/development.rb files will have to do for the moment.

Auto-completer for my blog comments

It took 4 hours to return from the town of Strängnäs to Stockholm via train. Normally, its 40 minutes. The train never turned up. 4 * 60 – 40 = 200 minutes of non-travel.

Stuck in a train station for 200 minutes I tackled a problem that has irked me for some time.

In my blog comments, when I reply to someone else’s comment I’ll normally use the syntax: “@chris – thanks for the kind feedback, I’ll send the money via paypal“.

And then Chris says “@Dr Nic – yeah, send it to the usual account; the tax man will never find it“.

And then I say “@chris – no problem“.

Its not that “@chris” is difficult to spell, but I’ve used IRC and Gmail long enough that I like auto-completion. The absence of auto-completion in Skype irks me too.

What I really want is:

“@c” + TAB expands to “@chris – “

So that’s what I now have. And its neat.

Demo of auto-completer

Unfortunately, the first person to leave a comment below won’t have anyone to reply to, and so theoretically wouldn’t get to experience the joys of the feature, that is so awesome and sexy that my very own wife called:

“That’s nice dear, now help with dinner”.

So, by default “Nic”, “Dr Nic”, “drnic” are available. The 2nd commenter will get these + the name of the 1st commenter. The 3rd commenter will get… hehe, you thought I was going to type that out.

Here’s what you’ll see when you save your comments:

Testing Autocompleter

Public release

Its currently implemented for WordPress, using Prototype/Scriptaculous. Well it works on my WordPress theme. It figures out the auto-completion list from the HTML, not from the blog/forum server.

I’m not sure how different everyone’s HTML is for their comment blocks. That makes it tricky to release the conde at the moment for general consumption. You might be an HTML guru and make it work, but your sister isn’t.

So, can you please comment below (wink wink) with the name of your blog software, and include your blog url in the URL field. I might be asking for grief here, but I’ll scope out everyone’s blog comment HTML and see how different/similar they are.


If someone in the comments has already mentioned your blog software (WordPress, Mephisto, etc) then you MUST reply to their comment. If you are first to comment with your blog software, then use “@Dr Nic” instead.

This is life-or-death critical as it gives you an excuse to play with the auto-completer thingy. Lives are at stake here people!

Don’t have a blog, but want to comment anyway to play with it in all its snazziness? Go for gold.