Dr Nic

I used to love Javascript so much that it would scare me. Why? Because I used to never write any tests. No unit tests. No integration tests (e.g. Selenium). Why? I didn’t know how. Not properly. I didn’t know how to write Javascript unit tests with an autotesting tool, like ZenTest’s autotest command.

And now…

I am now free to write as much Javascript as I like knowing forever I’ll write Javascript tests first, and Javascript in libraries second. TDD for Javascript beckons!

Below is a “getting started” tutorial, a helpful autotesting plugin, and hints about a TextMate bundle for Javascript Unit Testing and a future Peepcode on Javascript Unit Testing.

Installation

Fundamentally, the ideas here are app server-neutral. We’re using the unittest.js library from scriptaculous. But here I’ll show/demo the Rails plugins that make this all uber easy to get started.

Firstly, create a blank rails app, or use an existing app. The tutorial should be non-invasive - just a few “demo” files to delete when you’re done. This tutorial is independent of your app.

ruby script/plugin install javascript_test
mkdir test/javascript
ln -s ../../vendor/plugins/javascript_test/assets/ test/javascript/assets

For Windows people copy vendor/plugins/javascript_test/assets into test/javascript/assets, as you don’t have symbolic links, which is sad.

At this stage, you can add and commit these files to your project. The rest of this article is destructible demo files and tests.

Creating javascript tests

Let’s say

$ ruby script/generate javascript_test maths
    exists  test/javascript
    exists  public/javascripts
    create  test/javascript/maths_test.html
    create  public/javascripts/maths.js

maths.js is just a blank javascript file. The important file is maths_test.html. Its packed with lots of “getting started” goodies.

Open test/javascript/maths_test.html and find the testTruth method.

testTruth: function() { with(this) {
  assert(true);
}}

This is an example of a javascript test method, with a sample assert call. It has the same api as the Ruby test::unit assert call. Thomas Fuchs’ presentation Adventures in JavaScript testing gives a good rundown of the available assert methods (and the BDD style syntax that is available). I’m also preparing a TextMate bundle to generate the assert calls, with the same “as”, “ase”, “asnn” tab completions as the Ruby bundle. It makes me happy when using it.

Running unit tests

The most holistic way to run your javascript unittests, is against all the browsers you have on your local machine. The javascript_test plugin comes with a rake task for this: test:javascripts

$ rake test:javascripts
(in /Users/nicwilliams/Documents/rails_apps/imindi_theme)
/test/javascript/maths_test.html on Safari: SUCCESS
/test/javascript/maths_test.html of Firefox: SUCCESS
Skipping Internet Explorer, not supported on this OS
Skipping Konqueror, not supported on this OS

In the browsers you’ll see:

It runs all your tests on all browsers. That’s great for a Continuous Integration process, but for me, coming from the ZenTest world of autotest, I only want to run those tests for which something has changed.

I want continual feedback. Small change, run small test. autotest gives me that for Ruby, but rake test:javascript doesn’t.

Autotesting javascript tests

On my Mac, I’ve set up something as a starting point for autotesting javascript tests, based on the “small change, run small test” principle.

$ ruby script/plugin install http://drnicutilities.rubyforge.org/svn/plugins/javascript_test_autotest/
.
.
Edit config/javascript_test_autotest.yml for the browser(s) to use for autotesting.

Open config/javascript_test_autotest.yml and uncomment the browser(s) you want to autotest with. I autotest on Safari on the Mac, even though I develop/debug on Firefox, because it reloads each page in the same tab, which I like, and I’m having problems getting this to work into Firefox at all.

My yml file looks like:

browsers:
  safari: '/Applications/Safari.app/Contents/MacOS/Safari'

So, the current version is known to work for Mac OS X and Safari. If you get this plugin working on different O/S and browsers, let me know below or please submit a patch to http://groups.google.com/group/drnicutilities

From a command line:

$ script/js_autotest
Watching public/javascripts/controls.js, public/javascripts/application.js, test/javascript/maths_test.html, public/javascripts/effects.js, public/javascripts/dragdrop.js, public/javascripts/prototype.js

Files: 6

Let’s TDD with js_autotest

In test/javascript/maths_test.html, replace testTruth, with the following and save:

// Maths class should exist
testMathsClassShouldExist: function() { with(this) {
  assert(Maths, 'Where is Maths?');
}},

js_autotest will automatically load maths_test.html into your browser, and its tests executed.

Now, let’s fix the problem by creating the Maths javascript class. In public/javascripts/maths.js, add:

var Maths = Class.create();
Maths.prototype = {
  initialize: function() {

  }
}

js_autotest will again automatically reload maths_test.html into your browser, and its tests executed.

Yay for TDD and Yay for “small change, run small test”.

Peepcode

I love Geoffrey’s Peepcodes. Geoffrey has all my money in Peepcode credits. More importantly, Geoffrey’s due to give birth this year to #1 child. Geoffrey, who works from home, is blissfully unaware that soon he’ll be a full-time father and part-time Peepcode author. This could mean fewer peepcodes on important peepcode-worthy topics.

I always wished there was a Peepcode on Javascript Unit Testing. It would take many blog posts to cover as much as can be covered in a single 60 minutes peepcode. So, we’re writing/videoing one.

As a Javascript developer [has anyone ever heard of a Javascript Users Group?] let me know what specifically you’d like covered in this video, so I don’t accidently miss something.

[optional] Install some patches

There was a section here about some patches that could be applied. These have now been applied, and javascript_test plugin has been updated with prototype 1.6.0.1. Life is good.

I’m still trying to justify my effort writing the MagicCGI code. It let you get an XML or JSON feed for any database, with some basic conditionals, limits etc. The existing demo is for my blog database.

I think this one is kinda cool - showing off all your blog posts/articles on a timeline:

To get the timeline working with the schema output from the Magic CGI, I needed to write my own Timeline EventSource. Currently I don’t do anything fancy with the generated bubbles - I just use the defaults.

The MagicCGI query gets all wp_posts (Wordpress schema) rows, where “post_status=publish” and only returns fields that are relevant (notably ignores the large post_content field holding the blog content):

http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?
format=json&table=wp_posts&post_status=publish&
field=ID&field=post_title&field=post_date_gmt&
field=post_modified_gmt&field=guid&field=comment_count

The other cool thing I did here was to deploy it all with Capistrano (an html page + javascript libs). It even deploys/manages a copy of the Timeline trunk onto the server. This is the first time I’ve deployed a non-Ruby/Rails app using Capistrano, and once I got it set up it becomes much easier to manage than using an FTP app, etc.

How easy? cap1 update (note the dubious use of capistrano 1… I still… haven’t… converted… to 2.0…)

I’m not confident enough that my solution is sexy enough to outline in detail, so if you’re interested in deploying Javascript apps etc with Capistrano, just checkout the code (below) and look at the config/deploy.rb script. (note that I’ve disabled the deprec require statement as it assumes I’m wanted to run some mongrels etc, but deprec is very handy for setting up ssh and other fun stuff at the start).

So, no details here, just a fun example.

If the Timeline tickles your fancy, their website has lots of tutorials, and/or check out my html/javascript code.

svn co http://drnicwilliams.com/svn/blog_timeline/trunk blog_timeline

If the MagicCGI tickles your fancy, its also only on svn at the moment, though its docco should be pretty good. I think.

svn co http://rubyforge.org/var/svn/magicmodels/magic_cgi/trunk magic_cgi

In the last 20 days, 43 people have used OpenID to leave comments. That’s very cool.

Corollary: add OpenID login to your blog.

Even cooler - Dynamic counter

The screen shot comes from the Comments form.

If you look at the comments form in a week, month, year, the counter above have be changed from its original value 43.

No fancy Wordpress plugins (I don’t do PHP)

No Apache tricks. (I don’t know any Apache tricks)

Instead with Javascript/HTML attached to JSON attached to a RubyCGI script attached to my Wordpress database via ActiveRecords and some magic.

I call it MagicCGI. I also call it Frigging Scary.

Try the following:

$ curl -v "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_openid_identities&action=count&format=json"
< Content-Type: txt/json
<
43

Or some XML?

$ curl -v "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_openid_identities&action=count&format=xml"
< Content-Type: txt/xml
<
<?xml version="1.0" encoding="UTF-8"?>
<result>
  <count type="integer">43</count>
</result>

Raw data?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_posts&field=post_title&limit=5&order=post_title"
[{"attributes": {"post_title": "OpenID count"}},
 {"attributes": {"post_title": "MagicCGI shows OpenID user count"}},
 {"attributes": {"post_title": "map_by_method now works with ActiveRecord associations"}},
 {"attributes": {"post_title": "Feedburner"}},
 {"attributes": {"post_title": "One year on the InterTubes"}}
]

Getting kinda scary now, I think.

Ooh, just how much magic?

Want a list of available tables to play with?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?meta=tables"
[{table_name: 'wp_users'},...]

• add &format=xml to XML output; JSON is default
• add &meta=columns to include the column schema definitions
• add &table_name=wp_posts for each table you want (instead of all the tables)

E.g. to see the columns for wp_posts and no other table, in XML:

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?meta=tables&meta=columns&table_name=wp_posts&format=xml"

A list of all urls and internal user_ids for users/OpenID users/registered commenters?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_users&field=user_url&field=id"
[{"attributes": {"id": "1", "user_url": "http://drnicwilliams.com"}}, ...

So, now we know Dr Nic = user id 1.

Oh oh oh, how about a list of comments for a specific user?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_comments&user_id=1"
...comments by Dr Nic...
or
$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_comments&user_id=1&action=count"
232

Dr Nic’s commented in his own blog 232 times? Out of how many comments?

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_comments&action=count"
963

Where are the user emails? Where are their passwords?

Hidden! See below.

Callbacks?

JSON URLs support callback=someCallbackMethod and/or variable=someLocalVariable.

$ curl "http://drnicwilliams.com/cgi-bin/wp_drnicwilliams.cgi?table=wp_comments&action=count&callback=someMethod"
someMethod(963);

What does wp_drnicwilliams.cgi look like?

Something a little like this…

#!/usr/local/bin/ruby
require 'magic_cgi'         # loads the render magic, model magic, and meta-model magic
include Render

require 'magic_cgi/config/wordpress'  # connect to DB using Wordpress installation (wp-config.php)
MagicCGI::Config::Wordpress.establish_connection "/path/to/drnicwilliams/web/public"

# The following is defaulted for Wordpress connections:
MagicCGI::Config.hidden_tables |= %w[wp_openid_nonces wp_openid_associations wp_usermeta wp_tla_rss_map wp_tla_data]
MagicCGI::Config.hidden_columns['wp_users'] = %w[user_email user_pass user_activation_key]
MagicCGI::Config.hidden_columns['wp_openid_identities'] = %w[hash]
MagicCGI::Config.hidden_columns['wp_comments'] = %w[comment_author_email comment_author_IP]
MagicCGI::Config.hidden_columns['wp_posts'] = %w[post_password]

render do |params|
  data = DbTable.from_params(params)
  data ||= begin
    table_name = params['table'].first || 'wp_users'
    model_name = ActiveRecord::Base.class_name(table_name)
    klass = MagicCGI::MagicModel.create_class(model_name, ActiveRecord::Base)
    klass.find_or_count_by_params(params)
  end
  data
end

Can I write my own CGI scripts for my own DBs?

Sure. Its a library called MagicCGI.

What is MagicCGI?

Coming soon.

It took 4 hours to return from the town of Strängnäs to Stockholm via train. Normally, its 40 minutes. The train never turned up. 4 * 60 - 40 = 200 minutes of non-travel.

Stuck in a train station for 200 minutes I tackled a problem that has irked me for some time.

In my blog comments, when I reply to someone else’s comment I’ll normally use the syntax: “@chris - thanks for the kind feedback, I’ll send the money via paypal“.

And then Chris says “@Dr Nic - yeah, send it to the usual account; the tax man will never find it“.

And then I say “@chris - no problem“.

Its not that “@chris” is difficult to spell, but I’ve used IRC and Gmail long enough that I like auto-completion. The absence of auto-completion in Skype irks me too.

What I really want is:

“@c” + TAB expands to “@chris - “

So that’s what I now have. And its neat.

Unfortunately, the first person to leave a comment below won’t have anyone to reply to, and so theoretically wouldn’t get to experience the joys of the feature, that is so awesome and sexy that my very own wife called:

“That’s nice dear, now help with dinner”.

So, by default “Nic”, “Dr Nic”, “drnic” are available. The 2nd commenter will get these + the name of the 1st commenter. The 3rd commenter will get… hehe, you thought I was going to type that out.

Here’s what you’ll see when you save your comments:

Public release

Its currently implemented for Wordpress, using Prototype/Scriptaculous. Well it works on my Wordpress theme. It figures out the auto-completion list from the HTML, not from the blog/forum server.

I’m not sure how different everyone’s HTML is for their comment blocks. That makes it tricky to release the conde at the moment for general consumption. You might be an HTML guru and make it work, but your sister isn’t.

So, can you please comment below (wink wink) with the name of your blog software, and include your blog url in the URL field. I might be asking for grief here, but I’ll scope out everyone’s blog comment HTML and see how different/similar they are.

BUT!…

If someone in the comments has already mentioned your blog software (Wordpress, Mephisto, etc) then you MUST reply to their comment. If you are first to comment with your blog software, then use “@Dr Nic” instead.

This is life-or-death critical as it gives you an excuse to play with the auto-completer thingy. Lives are at stake here people!

Don’t have a blog, but want to comment anyway to play with it in all its snazziness? Go for gold.

Jesse Newland got the JSON API a day ago and already built a reusable Javascript widget to show conference session selections on his blog sidebar.

So I added it to my blog too!

Feel free to use his code, and write your own CSS to make it look sexy on your site.

Remember, its dynamic JSON (or XML) - so any changes you make to your conf plan will automatically show up next time the widget is displayed.

Doubly remember - use your JSON url not Jesse’s… its the link “json” at the top of each conference page, when you’re logged in. Alternately, just change jnewland to your username. Yeah, that’s simpler.

[Most of this text copied from original announcement]

Thanks Jesse!