Dr Nic » Greasemonkey

Nifty Threaded IM Chat within Gtalk/Gmail Chat

Dr Nic — Fri, 03 Jul 2009 20:00:54 +0000

Ever had IM chats where a conversation splits into multiple topics? You’ll be able to follow along, intelligently piecing together which-message-goes-with-which-topic, until the following scenario inevitably occurs:

  me: What's on this weekend? Going to the football?
  me: Are you and Jackie still seeing each other?
  you: Yes
  me: Eh? Yes - football or yes to Jackie?

That is, eventually the messages become ambiguous as to which topic they go to.

The solution

A designer friend of mine and I discovered this problem every day as we talked about different projects and completely unrelated things. Ironically, this led to a new inline topic: what if each thread/topic could be visually identifiable?

Perhaps we could just modify one of the HTML-based IM clients, such as Gmail Chat/Gtalk (same could be done for Facebook’s IM client I guess), and use twitter-esque #tags to identify threads (no fancy jabber protocol changes). If we did this we could prototype something, see if it was a useful way to solve the multi-threaded IM chat problem. I mean, how hard could it be?

Prototype: Greasemonkey Script (Firefox + Safari)

Since I have a thing for Greasemonkey scripts at the moment (which also run on Safari/Mailplane using GreaseKit), it immediately came to mind as a way to hack into Gmail’s Chat.

After installing Greasemonkey or GreaseKit, click to install the extension for Gmail’s inline Jabber/Gtalk chat.

Restart Gmail, fire up a chat to someone (for example, complain of bugs to drnicwilliams@gmail.com) and try the following:

  greasemonkey is fun
  gmailchat is very nifty and hackable
  its cool that I can annotate gmail chat with #greasemonkey
  no way, #gmailchat is colour highlighted
  sexy

Which will look something like the picture at the top.

Sadly, I’m talking to myself here. QA testing can be a lonely man’s sport.

Success?

Technically, yes. I mean, it works. You use a different #tag and it will be a different colour.

It was a prototype to determine if using #tags was a friendly, non-invasive way to identify threads. And it kind-of works, as long as you remember to use them. In IM, less-so than twitter, it seems unnatural to add #tags, or prefix a keyword with a # character. But, in time, I think you’d learn to do it to get the benefit.

The bigger issue is that I don’t want to use Gmail’s Chat for my IM client. I didn’t find the source to Apple’s iChat client lying around on github; and I really don’t want to go hacking Cocoa/Win32 apps just to try out an idea. A greasemonkey script is an awesome way to try out something like this.

Now, if everyone could just make this idea of #tagging intra-IM conversation threads/topics, then perhaps 5 years from now Apple will pick it up and implement it in iChat. Any of the more accessible, open source clients could implement this too. Probably a lot sooner.

Known bugs

In Mailplane (though not Safari), the 2nd+ threads aren’t coloured differently. I’m having trouble fixing this at the moment due to an issue in blue-ridge’s setup on Safari.

It currently shares thread colouring across all open chat windows. Probably not a big issue. I forgot to consider multiple chat windows when I wrote the code.

When you use a new #tag, it only finds one previous message with ‘tag’ in it. Really, once a word is #tagged, then any message containing ‘tagged’ could be included in the thread.

Quirky thing I learned

You can’t really use the jquery.livequery.js plugin to watch for DOM changes in Greasemonkey scripts. It works by hooking into jQuery DOM modification calls, such as append and prepend, to know instantly that something has changed. Gmail, and many other websites, don’t use jQuery. So it doesn’t work. Even though your beautiful unit tests say it will. Use setInterval instead.

Project status

It’s finished. It was a prototype to try out an idea. It has unit tests, it works and if you want to use it for your own research project or “oh oh oh how cool would it be if…?” hackathon, go for gold with the code base. Rename it, abuse it. Have fun.

Source on github: http://github.com/drnic/threaded-gtalk-gmscript/

A pleasant word from my sponsor

When I was hacking on Threaded Gtalk GMScript, I wasn’t doing something more productive at my firm Mocra. Ironically, you could reward my wayward efforts by considering us for your awesome new Rails or iPhone project. It will make you happy. Especially if its chock-full of JavaScript.

TDD for Greasemonkey scripts; and introducing Ninja Search JS “this article shows how I used test-driven development tools and...
Prototype: “element-id”.$() instead of $(‘element-id’) The Prototype library gives us the $() operation for converting...
Yehuda Katz starts a blog Yehuda is the creator of autoDB – the wonderful admin...

TDD for Greasemonkey scripts; and introducing Ninja Search JS

Dr Nic — Sat, 06 Jun 2009 23:00:40 +0000

“this article shows how I used test-driven development tools and processes on a Greasemonkey script.” Though it also includes free ninjas.

When I do online banking I need to select from a large list of other people’s bank accounts to which I might like to transfer money too. It is the massive drop down list that I must scroll through that I wish to raise issue with today. The problem of having to give other people money is probably a different discussion.

And take those time-zone selector drop down lists, for example, the massively long list rendered by Rails’ time_zone_select helper. Granted, I am thankful for you letting me choose my timezone in your web app. Though for those of us not living in the USA we must hunt for our closest city in the list. Dozens of locations, ordered by time zone and not the name of the city (see adjacent image). Unfortunately you can’t easily type a few letters of your current city to find it. Rather, you have to scroll. And if you live in the GMT+1000 time zone group (Eastern Australia), you have to scroll all the way to the bottom.

So I got to thinking I’d like a Greasemonkey (for Firefox) or GreaseKit (for Safari) script that automatically converted all ridiculously long HTML drop down lists into a sexy, autocompletion text field. You could then type in “bris” and be presented with “(GMT+1000) Brisbane”, or given the less amusing banking scenario then I could type “ATO” and get the bank account details for the Australian Tax Office.

I mean, how hard could it be?

This article is two things: an introduction to Ninja Search JS which gives a friendly ninja for every drop down field to solve the above problem. Mostly, the rest of this article shows how I used test-driven development tools and processes on a Greasemonkey script.

Introducing Ninja Search JS

Click the banner to learn about and install the awesome Ninja Search JS. It includes free ninjas.

Currently it is a script for Greasemonkey (FireFox) or GreaseKit (Safari). It could be dynamically installed as necessary via a bookmarklet. I just haven’t done that yet. It could also be a FireFox extension so it didn’t have to fetch remote CSS and JS assets each time.

Ninja Search JS uses liquidmetal and jquery-flexselect projects created by Ryan McGeary.

Most importantly of all, I think, is that I wrote it all using TDD. That is, tests first. I don’t think this is an erroneous statement given the relatively ridiculous, and unimportant nature of Ninja Search JS itself.

TDD for Greasemonkey scripts

I love the simple idea of Greasemonkey scripts: run a script on a subset of all websites you visit. You can’t easily do this on desktop apps, which is why web apps are so awesome – its just HTML inside your browser, and with Greasemoney or browser extensions you can hook into that HTML, add your own DOM, remove DOM, add events etc.

But what stops me writing more of them is that once you cobble together a script, you push it out into the wild and then bug reports start coming back. Or feature requests, preferably. I’d now have a code base without any test coverage, so each new change is likely to break something else. Its also difficult to isolate bugs across different browsers, or in different environments (running Ninja Search JS in a page that used prototypejs originally failed), without a test suite.

And the best way to get yourself a test suite is to write it before you write the code itself. I believe this to be true because I know it sucks writing tests after I’ve writing the code.

I mostly focused on unit testing this script rather than integration testing. With integration testing I’d need to install the script into Greasemonkey, then display some HTML, then run the tests. I’ve no idea how’d I’d do that.

But I do know how to unit test JavaScript, and if I can get good coverage of the core libraries, then I should be able to slap the Greasemonkey specific code on top and do manual QA testing after that. The Greasemonkey specific code shouldn’t ever change much (it just loads up CSS and more JS code dynamically) so I feel ok about this approach.

For this project I used Screw.Unit for the first time (via a modified version of the blue-ridge rails plugin) and it was pretty sweet. Especially being able to run single tests or groups of tests in isolation.

Project structure

All the JavaScript source – including dependent libraries such as jquery and jquery-flexselect – was put into the public folder. This is because I needed to be able to load the files into the browser without using file:// protocol (which was failing for me). So, I moved the entire project into my Sites folder, and added the project as a Passenger web app. I’m ahead of myself, but there is a reason I went with public for the JavaScript + assets folder.

In vendor/plugins, The blue-ridge rails plugin is a composite of several JavaScript libraries, including the test framework Screw.Unit, and a headless rake task to run all the tests without browser windows popping up everywhere. In my code base blue-ridge is slightly modified since my project doesn’t look like a rails app.

Our tests go in spec. In a Rails app using blue-ridge, they’d go in spec/javascripts, but since JavaScript is all we have in this project I’ve flattened the spec folder structure.

The website folder houses the github pages website (a git submodule to the gh-pages branch) and also the greasemonkey script and its runtime JavaScript, CSS, and ninja image assets.

A simple first test

For the Ninja Search JS I wanted to add the little ninja icon next to every element into a text field with fantastical autocompletion support.

For Screw.Unit, the first thing we need is a spec/ninja_search_spec.js file for the tests, and an HTML fixture file that will be loaded into the browser. The HTML file’s name must match to the corresponding test name, so it must be spec/fixtures/ninja_search.html.

For our first test we want the cute ninja icon to appear next to

In its header it loads the blue-ridge JavaScript library, which in turn loads Screw.Unit and ultimately our spec.js test file (based on corresponding file name), so ninja_search.html will cause a file spec/ninja_search_spec.js to be loaded.


To run our first test just load up the spec/fixtures/ninja_search.html file into your browser.
Your first test will fail. But that’s ok, that’s the point of TDD. Red, green, refactor.
Simple passing code
So now we need some code to make the test pass.
Create a file public/ninja_search.js and something like the following should work:
(function($){
  $(function() {
    $('select').each(function(index) {
      var id = $(this).attr('id');

      // create the Ninja Search button, with rel attribute referencing corresponding >select id="...">
      $('> class="ninja_search_activation" rel="' + id + '">ninja search>/a>')
      .insertAfter($(this));
    });
  });
})(jQuery);

Reload your test fixtures HTML file and the test should pass.
Now rinse and repeat. The final suite of tests and fixture files for Ninja Search JS are on github.
Building a Greasemonkey script
Typically Greasemonkey scripts are all-inclusive affairs. One JavaScript file, named my_script.user.js, typically does the trick.
I decided I wanted a thin Greasemonkey script that would dynamically load my ninja-search.js, and any stylesheets and dependent libraries. This would allow people to install the thin Greasemonkey script once, and I can deploy new versions of the actual code base over time without them having to re-install anything.
Ultimately in production, the stylesheets, images, and JavaScript code would be hosted on the intertubes somewhere. Though during development that would be long-winded and painful to push the code to a remote host just to run tests.
So I have three Greasemonkey scripts:

public/ninja_search.dev.user.js – loads each dependent library and asset from the local file system
public/ninja_search.local.user.js – loads compressed library and asset from the local file system
public/ninja_search.user.js – loads compressed library and assets from remote server

Let’s ignore the optimisation of compressing dependent JavaScript libraries for the moment and just look at the dev.user.js and user.js files.
The two scripts differ in the target host from which they load assets and libraries. ninja_search.dev.user.js loads them from the local machine and ninja_search.user.js loads them from a remote server.
For example ninja_search.dev.user.js loads local dependencies like this:
require("http://ninja-search-js.local/jquery.js");
require("http://ninja-search-js.local/ninja_search.js");

And ninja_search.user.js loads remote dependencies like this:
require("http://drnic.github.com/ninja-search-js/dist/jquery.js");
require("http://drnic.github.com/ninja-search-js/dist/ninja_search.js");

In the final version of ninja_search.user.js we load a simple, conpressed library containing jquery, our code, and other dependencies, called ninja_search_complete.js.
Using Passenger to server local libraries
The problem with loading local JavaScript libraries using the file:// protocol, inferred earlier, is that it doesn’t work. So if I can’t load libraries using file:// then I must use the http:// protocol. That means I must route the requests through Apache/Ningx.
Fortunately there is a very simple solution: use Phusion Passenger which serves a “web app’s” public folder automatically. That’s why all the javascript, CSS and image assets have been placed in a folder public instead of src or lib or javascript.
On my OS X machine, I moved the repository folder into my Sites folder and wired up the folder as a Passenger web app using PassengerPane. It took 2 minutes and now I had http://ninja-search.local as a valid base URL to serve my JavaScript libraries to my Greasemonkey script.
Testing the Greasemonkey scripts
I can only have one of the three Greasemonkey scripts installed at a time, so I install the ninja-search.dev.user.js file to check that everything is basically working inside a browser on interesting, foreign sites (outside of the unit test HTML pages).
Once I’ve deployed the JavaScript files and assets to the remote server I can then install the ninja-search.user.js file (so can you) and double check that I haven’t screwed anything up.
Deploying via GitHub Pages
The normal, community place to upload and share Greasemonkey scripts is userscripts.org. This is great for one file scripts, though if your script includes CSS and image assets, let alone additional JavaScript libraries, then I don’t think its as helpful, which is a pity.
So I decided to deploy the ninja-search-js files into the project’s own GitHub Pages site.
After creating the GitHub Pages site using Pages Generator, I then pulled down the gh-pages branch, and then linked (via submodules) that branch into my master branch as website folder.
Something like:

git checkout origin/gh-pages -b gh-pages
git checkout master
git submodule add -b gh-pages git@github.com:drnic/ninja-search-js.git website

Now I can access the gh-pages branch from my master branch (where the code is).
Then to deploy our Greasemonkey script we just copy over all the public files into website/dist, and then commit and push the changes to the gh-pages branch.

mkdir -p website/dist
cp -R public/* website/dist/
cd website
git commit -a "latest script release"
git push origin gh-pages
cd ..

Then you wait very patiently for GitHub to deploy your latest website, which now contains your Greasemonkey script (dist/ninja-search.user.js) and all the libraries (our actual code), stylesheets and images.
Summary
Greasemonkey scripts might seem like small little chunks of code. But all code starts small and grows. At some stage you’ll wish you had some test coverage. And later you’ll hate yourself for ever having release the bloody thing in the first place.
I wrote all this up to summarise how I’d done TDD for the Ninja Search JS project, which is slightly different from how I added test cases to _why’s the octocat’s pajamas greasemonkey script when I first started hacking with unit testing Greasemonkey scripts. The next one will probably be slightly different again.
I feel good about the current project structure, I liked Screw.Unit and blue-ridge, and I’m amused by my use of GitHub Pages to deploy the application itself.
If anyone has any ideas on how this could be improved, or done radically differently, I’d love to hear them!
Polish your Rails project with Mocra
I want to help you, your business, your boss and your project reach delightful levels of wickedly awesomeness. I’m so proud of the small team of ace Rails developers here at Mocra and what I know we can do for you.
Send an email to rails@mocra.com about your current/future projects. Dare us to be more awesome!
While you wait for a reply perhaps you’d like to learn more about How we do it at Mocra?


Related posts:
Using CoffeeScript in Rails and even on Heroku I’m pretty excited about CoffeeScript as a clean-syntax replacement for...
Dead simple JavaScript Unit Testing in Rails  Formats: Video/Screencast (410 Mb, torrent) | Video only (vimeo)...
Nifty Threaded IM Chat within Gtalk/Gmail Chat Ever had IM chats where a conversation splits into multiple...