Continuous integration in web development

Leave a reply

CI, or Continuous Integration, is a big help when you’re working on console applications. If you’re not familiar with the term, continuous integration refers to a system through which you can have your code compiled, executed, and tested in the background as you’re working on it. This usually happens on remote servers dedicated to the heavy-lifting involved with building code. You can be notified of build failures asynchronously and test new features without having to interrupt your workflow to wait for a compile job to finish.

However, things are different with front-end web development. Websites are hard to test, and when they are tested, they’re usually tested by hand in some kind of re-build/alt-tab/refresh loop, without the power of CI. Following are some tips I derived from CI that may help you improve your web development workflow.

If you’ve ever worked on a large JavaScript-intensive application, you’ve probably written build scripts to concatenate and minify JavaScript. You don’t need a huge IDE or development framework to accomplish any of that. I regularly use the following tools and scripts to help me develop my Final Grade Calculator:

CLI JavaScript Minification

There are plenty of websites that will do JS-minification for you online. In fact, most of them run in JavaScript themselves. However, if you find yourself visiting them more than once a day, you should get yourself a command-line JavaScript minifier. Dean Edwards, the author of /packer/, one of the most popular JS minification websites online, has ports of his JS packer available in several programming languages on his website. (I’m using the PHP version, because it appears to be the most faithful port.)

After you’ve acquired a minifier and have appended it to your $PATH, you can incorporate it into your build scripts like so:

#!/bin/bash
packer src/final.english.js out.js
./build.py > out.htm
...

inotify Tools

Even if you’ve combined all of your build tasks into a single build script, you still have to run the thing every time you want to see new changes in your web application. A sleep-build loop would take care of this inconvenience, but then you’re stuck between wasting CPU cycles and having to wait a bit for new changes to appear. We can do better.

The term inotify stands for index-node (inode) notification. Most unix systems come with tools that bridge inotify services with the CLI, and you can use these to have your build script run only when you change and save a source file in your text editor.

I have an alias set up like so:

alias watchdir='inotifywait -r -e close_write,moved_to,create'

The switches enable recursive behavior and restrict the events to a certain few. This command will block until one of the three events occurs:

  • close_write – when a file handle in w mode is closed
  • moved_to – when a file is moved into the directory
  • create – when a new file is created in the directory

Combined with an rsync alias with some reasonable defaults, you can put together a loop that syncs source files to a remote server and builds them as they change. 

alias rrsync='rsync -vauzh --progress --delete'

For example, I used something like the following while developing this blog’s theme:

while watchdir cobalt; do sass ... ; rrsync cobalt ... ; done

This last one isn’t a concept from CI, but it can be adapted to fit in your CI workflow whenever you need it.

Ad-hoc Web Server

You can test static html files just by opening them locally with your web browser, but there are a few reasons that an actual web server, no matter how simple it may be, is a slightly better option (root favicon, greater network permissions, absolute paths, protocol-relative URL’s, just off the top of my head). Python (and Python 3) comes with a built-in single-threaded simple web server that supports directory listing, symbolic links, a decent number of MIME types, modification/expiration headers, and large-file streaming. In other words, it’s a pretty good tool that will do anything you could want related to serving static assets (you know, unless you want to download two things at once).

Start the python web server at the command line, and it will start serving files through HTTP from the current working directory:

> python -m SimpleHTTPServer [port number]  # Python 2
> python3 -m http.server [port number]      # Python 3