Dr Nic

Migrating project websites to github pages with sake tasks, new websites with jekyll_generator

Its almost Christmas time and that means presents.

It also means that sometime between today and December 25th you need to go out and buy other people some presents because you haven’t done it yet.

But there’s someone else special in your life that deserves an Xmas gift this year. That special someone is your open source projects.

You might think, “But, Dr Nic, what do I get for the open source project that already has everything? What gift would my open source projects possible appreciate?”

Its true – this year has been a boon for open source. You probably migrated your projects from SVN to Git this year. Specifically, you probably clicked the “create a new one” link on your GitHub home page a lot this year: either to migrate your old projects to github or start new ones. Lots of new ones.

And now that your code is on github, your README file is rendered beautifully on the home page, so you were more inclined to rename it to README.markdown or README.textile and thusforthly fill it full of education information about your projects.

More people would then instantaneously figure out what the f@#$ your project actually does, thus more people used it, thus more people wrote patches or forked your code and sent Pull Requests.

All round, in 2008, I bet your open source projects had a sweet year.

Nonetheless, it is Christmas time and you should now get them a present. Something they wouldn’t get for themselves. Something you wouldn’t have done for them except for the free loving spirit of Christmas.

The Gift for all Open Source projects

A website.

Oh sure, github renders your README.markdown file. Yeah, yeah, github gives you a wiki for your project. Sure, sure, Google Groups let you communicate with your co-developers and users. Certainly, you don’t need a website for your project.

That’s why its a Christmas gift. You’re going to do it because you care.

Websites sell your project. READMEs and Wikis educate. The project website will sell it and make people want to use your stuff.

A website could have a blog with an RSS/atom feed. Then you could post updates about your project. People could subscribe and also leave comments. Oh the novelty.

And if it only took 5 minutes to get all this setup – the website code, the blog engine, the RSS feed, the comments, and published to its own hosted server – then that would just be oh so sweet.

So we’re going to do some craft for our Xmas present. A little DIY project, if you will. You’ll need a few things that you’ll find around your house, a command-line interface, and a beer or perhaps some port or sherry. Christmas cake is good too.

The commands below take about 5 minutes to execute and you’ll have a sweet website for your project, setup on GitHub Pages, just like the macruby-tmbundle and jekyll_generator sites, and the sample image above.

Jekyll and GitHub Pages

Jekyll, by Tom Preston-Werner, is one of several static website generators. All your website content will be written in textile or markdown, rendered against some HTML layouts, and published on a website somewhere. We’re going to use GitHub Pages because they are new and shiny, and free, and I’ve written some sake tasks (below) to make it all easy peasy to get up and running.

There are a bunch of reasons we’re going to use Jekyll to render our website:

One, textile and markdown are much nicer to write/edit text than HTML.

Two, Jekyll is “blog enabled”. That is, some of your pages will be specifically “blog post” pages, and elsewhere in your layout you can programmatically list them. Like a blog. Yeah.

Three, GitHub Pages automatically supports Jekyll but no other static-website generators. This saves you from having to manually render your HTML from your Textile/Markdown pages before pushing the site to the remove server.

Four, I wrote a stand-alone generator to create a Jekyll website for any project. It is preconfigured with Disqus for comments, including badges for latest comments etc.

Living with a Jekyll

Jekyll comes as a RubyGem:

sudo gem source -a http://gems.github.com/
sudo gem install mojombo-jekyll

You can either read the Jekyll README and create your project website by hand (and then skip below to Living with GitHub Page).

Alternately, you can cheat and use a generator.

jekyll_generator

For the last few years, newgem has come with the ability to create a website for your project. For the jekyll_generator I decided to go with a separate generator so that any project – rubygem, rails plugin, javascript project, etc – can easily use it.

sudo gem install jekyll_generator
cd path/to/project
jekyll_generator website --title "Project Name"
cd website
jekyll
open _site/index.html

The final steps performs the local rendering of your website (jekyll) and then on OS X it opens the rendered website into your browser. Otherwise, manually open the index.html from the _site folder.

Currently the generator assumes that your project is already in a git repository that is hosted on github (specifically that you’ve added the github project as the origin remote).

Living with GitHub Pages

GitHub recently released GitHub Pages as a place for you to publish a personal website and websites for each of your projects. Instead of requiring you to rsync or “rake website” your way to publishing glory, you just use git.

Specifically, you put your website in a branch called “gh-pages”. When you push commits in that branch to github it will automatically refresh your website. No rsyncing required. Also, it will run your website through Jekyll.

So much delicious magic.

Except our website is not currently in another branch. Its in master. In a folder called website. Where it should be.

But the boffins at github say we need to move our shit across to a branch called gh-pages, and who are we to argue.

I personally will never be doing this migration manually. I figured it out once and slapped it into a sake task. It moves all the content in your website folder into this branch, and replaces it with a submodule link so you can access the website from your master branch. Didn't follow that? No, me neither. Let's just run the sake task and move on with our lives, and eat fresh prawns and Christmas pudding.

Sake tasks to get started with GitHub Pages

To install them from scratch (initial blog post):

sudo gem install sake
cd /tmp
git clone git://github.com/drnic/sake-tasks.git
cd sake-tasks
rake install:all

You can toss away the repo after installing the sake tasks.

You now have some tasks to manage github:pages:

sake github:pages:migrate_website   # Migrates an existing website folder into a gh-pages branch, and links back as submodule
sake github:pages:setup             # Creates the gh-pages branch, and links to it as 'website' as submodule

Since we've already created the website above, we're going to use the first task to migrate this folder into the required gh-pages branch.

sake github:pages:migrate_website

Um, and your done.

Note, if sake crashes and burns in your project it might be that you don't have a Rakefile. Try, "touch Rakefile" and rerun the sake command.

In 1-10 minutes your fancy new site will appear at http://username.github.com/projectname. It even has a complimentary first blog post. You might want to change it to something useful.

You can now update your website via your website folder, commit changes and they automatically appear, automatically rendered via Jekyll, on your website.

Disqus for comments

Since your new website is static text there is no way to support comments on the blog posts. I've bundled into the theme's layouts the access code for Disqus. Like the image above, each blog post has Disqus comments at the bottom, and the sidebar shows the latest comments from all your blog posts.

You need to create a Disqus account, and create a project with the same name as your github project name. For example, if jekyll_generator is the project name on github, then make it the name on Disqus.

In my example above, my github project name was jekyll_generator but I couldn't use underscores on Disqus, so it is jekyll-generator. Fortunately, the generator templates already know this, and have converted your underscored name to hyphens already.

You can skip the "Choose your platform to install" steps as this is all done by the generator templates.

If you reload your project's website it should have Disqus badges on the sidebar, and the posts should have comment boxes.

Lighthouse for tickets

Lighthouse offers free access for open source projects to track their bugs. After you've created your Lighthouse project, you need to change the TODO values in _layouts/default.html and _layouts/post.html to your Lighthouse project number.

Updating your website

Your website now appears within your project source under the website folder. Though if you change it, add new posts, etc, you won't be able to commit the changes back normally. The contents of the website folder are effectively another git repository.

To commit changes to your website, go into the website folder and proceed normally:

project(master) $ cd website
project/website(gh-pages) $ git commit -m "some changes"
project/website(gh-pages) $ git push origin gh-pages
project/website(gh-pages) $ cd ..
project(master) $

Now your submodule may be dirty and need updating:

git submodule update --init

Not sure what this is all about.

Summary

A website is a great way to show that your pride and joy isn't ignored by new visitors and reminds existing users just how awesome the project is, and by clever association, how awesome you are.

A sexy website has to be even better.

Jekyll is great, and the Github Pages concept seems fun. Perhaps it wouldn't be necessary if the Github Wiki content was available as its own git repo, or if you could theme your project's source browser like a diry MySpace page.

Hopefully the sake tasks make migration of any existing websites to the Github Pages system easy peasy, and hopefully the jekyll_generator is useful for any new projects.

Hopefully.

Merry Christmas.

Related posts:

  1. Hacking someone’s gem with github and gemcutter Ever used a rubygem, found a bug, and just...
  2. Future proofing your Ruby code. Ruby 1.9.1 is coming. Bugger. I’m a Ruby monogamist. I use the Ruby...
  3. newgem 1.0.0 all thanks to Cucumber The New Gem Generator (newgem) was exciting, moderately revolutionary, and...
  4. My attempt at sake task management I’ve used sake intermittently in my workflow. It competes...
  5. How to yell at people with GitHub from TextMate Sometimes when you are perusing code you ask the question:...

18 Responses to “Migrating project websites to github pages with sake tasks, new websites with jekyll_generator”

  1. [...] Dr Nic wrote up an awesome post: Migrating project websites to github pages with sake tasks, new websites with jekyll_generator [...]

  2. Hmm.

    $ jekyll_generator website –title “Model Plus”
    create
    create _posts
    create index.markdown
    create atom.xml
    create config.yml
    create _posts/2008-12-21-first-post.markdown
    No such file or directory – /Library/Ruby/Gems/1.8/gems/jekyll_generator-1.0.0/app_generators/jekyll_generator/templates/_posts/first_post.markdown

  3. Mislav says:

    You rock, mate! Can’t wait to try this out —

    Now we have GitHub for code, README rendering, wiki pages, gitrdoc.com (in the works), Lighthouse integration, personal pages and custom project pages … 2008 has truly been a great year for open-source!

  4. Dr Nic says:

    @mike – fixed. Sorry for that. I added that file at the last minuted and forgot to bump the manifest.

  5. [...] Migrating project websites to github pages with sake tasks, new websites with jekyll_generator – More very cool stuff from Dr. Nic. [...]

  6. pjmm says:

    Thanks: looks great… Small correction for us newbies: I think the line

    sudo gem source http://gems.github.com

    should read

    sudo gem source -a http://gems.github.com

    in order to actually push the given source onto the list of sources.

  7. Dr Nic says:

    @pjmm – thank, updated.

  8. Dr. Nic,
    I liked what you did with Jekyll. This is just wonderful.
    Happy New Year to you.. I mean to all of us.
    Sudhindra

  9. trans says:

    I was pleasantly surprised by the new Pages feature, but I was terribly disappointed to see it use a branch. I simply won’t let GitHub pollute my branches this way. Not only does it decouple the version of the website from that of the rest of the project, but it misrepresents the intent of a branch. For instance, consider if someone created an automated tool that reported on the differences between all the branches and the current master, the gh_pages branch would require special exception, in this case just to satisfy GitHub. So I’m glad to see a work around, unfortunately it look like it only mitigates the issue, rather than actually solves it. I will probably not bother and just host my site on rubyforge.org instead. But I would like to as least see the code for this. Do you think you could post the sake task as a Gist? Thanks.

  10. Peter Jaros says:

    @trans: Actually, that’s a pretty common thing to do. It might be better done as a non-branch ref, but being a branch isn’t that terrible. Remember, this is git: you don’t have to have the gh_pages branch in *your* repo for it to be in GitHub’s. :)

  11. kylemac says:

    This has been amazingly helpful, not only play with Jekyll but get a better understanding of github in general (as I am a noob).

    I ran into an issue with the sake migrations, likely due to something I did. My errors read:

    ! [rejected] gh-pages -> gh-pages (non-fast forward)
    error: failed to push some refs to ‘git@github.com:kylemac/kylemac.github.com.git’

    and

    rake aborted!
    can’t convert true into String

    —-any ideas here? any help would be amazing.

  12. Dr Nic says:

    @kyle – yeah I discovered that issue for user home pages recently. The generator was built for project pages (e.g. drnic.github.com/some_project) rather than drnic.github.com user pages. So the generator code needs fixing to support this.

    For the [rejected] issue – you find that you’ll forever have either master or gh-pages rejected, depending if you’re in root or the website folder. Its not an issue.

    To find out what is broken with a faulty rake command, pass –trace to get a stack trace. It will probably be issues relating to the first paragraph. Check the generated code that urls etc are correct. Ping again if you can’t figure it out.

  13. kylemac says:

    Thanks for the help, again awesome project and great tutorial – super helpful. (wow i use too many adjectives)

  14. Thomas says:

    The git submodule update --init is IIUC equivalent to


    git submodule init # register any new submodules in .gitmodules into .git/config
    git submodule update # update the local dir

    Very useful post, thanks very much.

    – Thomas

  15. Thanks! Recently I’ve been reading the RSpec beta book from Pragmatic Programmers, and it was not too much clear for me. Now I th

  16. I just happened to stumble on this blog while surfing. This is a very insightful article. I really appreciated the info.