I submitted my application entry, GithubFinder, to the 10K Apart Contest last night. GithubFinder is basically a Mac-like Finder app to explore a Github repository. A lot of the time when I’m visiting Github, I wanted to quickly browse a repository, but there isn’t a quick way to do so. The 10K Apart contest gave me a reason to build something cool that I also could use. The rules to the contest are: the app needs to be less than 10K in total and it has to be 100% client-side and 100% compatible with IE9, Firefox, and Chrome/Safari. That means everything, markup, images, stylesheets, and CSS, need to be less than 10 KB (10,240 bytes) and there won’t be any server-side processing for the app to work. Packing a high-impact application into 10,240 is surprisingly not simple, I’ve learned quite a few things working on GithubFinder.
I wanted to share a technique that was invented by NIHILOGIC. The idea is to store the javascript code inside a PNG file to leverage the image compression and significantly reduce the size. And this technique can also be applied to CSS as well. I hope that my implementation would benefit someone before the 10K contest end.
From the early stage of the app, I know that I’d want to go over the 10K limit (well, limits are there to be pushed, and rules are set to be broken!) I really wanted GithubFinder to be useful, even in a minimal form. Moreover, I’m not the briefest coder in the world, so I decided to cheat the 10K rule by using Nihilogic’s inventon. I had read his article a while back, and I was amazed by his 8KB Mario implementation. Since IE9 supports Canvas, now is the time to whip out this trick. However, coming back to the article, the script to encode the JavaScript was no longer available. I ended up spending a few hours to re-created the javascript-to-png script again in Ruby, and a long the way, wrote an automate build script for the application.
If there’s anything in this project I’m really proud of, probably it’s my build script. The build process for the application works like this: all JS and CSS files are merged and minified using YUICompressor, then the minified JS and CSS are concatenated into one string, separated by a unique delimiter. This long string is then encoded into a PNG image. If there is any error in the javascript, YUICompressor would prompt and I’d jump to the error line, fix, then re-run the build to verify. Here’s the first part of the build-script containing the utility methods to handle all the merging, compressing, and generating image.
The merging and minification are both done inline to not litter the application with temporary files. Using Open3, I was able to just use yuicompressor directly (line 19). The interesting bit is the string2png() and its sibling, and the string2pngs() method. Here’s the string2png() method again:
string2png would read the input string character by character and convert this character to its ASCII equivalent, then create a new pixel with the red color of the ASCII value (line4 and 5). Afterward, we compress the image using the ZipCompression and write it out as an 8-bit PNG file. If you open up this file in Photoshop, you’ll see a long vertical stripe of different shade of red. One gotcha for this technique is that you won’t be able to use UTF-8 characters since we don’t have ASCII equivalents for them.
On the front-end, I created a “bootstrap.js” script similar to Nihilogic’s one to read back in the image file. Essentially the bootstrap.js script would load up the image, paint it on the canvas, then read back the pixel one-by-one and grab the red color out. Since the characters are stored in binary format and compressed, the code can greatly be minified.
The image produced by string2png works well in Chrome and Firefox, however, with IE9, there is a bug preventing the image to be read back properly. IE9’s getImageData() could only read the first 8192 bytes of the canvas pixel data, everything after the 8192th pixel are zero’ed out (you can see my test cases here — IE9 would consistently fail). (SEE UPDATE 1 below for the fix) The work around is to split the image into smaller ones, each has a maximum of 8192 pixels. The bootstrapper’s job now is to read these image files sequentially back in before reconstructing the js/css string. Hence we have string2pngs method:
The down side of this IE9 work-around is that the compression efficiency is reduced, since intead of having just 1 image, we’d have 2 or 3 images, and they all have to store their own color palette separately as apposed to sharing a single one. I estimate that I lost about 20% of storage space(1 single image: ~9.5KB, 3 equivalent images: ~12KB) due to the the splits. I ended removing features so that I could fit everything into the 10K limit.
At any rate, here’s the bootstrap.js script to handle the decoding of the image:
The function of interests here is x(). The whole decoding businesss are done from line 26 to 29, then the decoded string are passed back to the callback. Please bear with me the short variables, I was trying to minify this script. The run() call back would check for the chunks, and only proceed when it receives all the chunks. My earlier naive implementation of run() had a bug causing by a race condition which the images were re-assembled in different orders, thus sometimes the app initialized properly, and some other times it would fail.
Together with the build script, I had a quite flexible mechanism to tweak and play with the output files to get to the targeted size. Click here to see my full build script. I hope this help people participating in the 10K contest to create more wonderful applications.
Finally, my verdict on IE9 Platform Preview release? Unimpressive and buggy. I was able to crash it reliably (!). Microsoft should have waited a few more weeks to put in more features before rushing to release a piece of inferior software. Even worse, they didn’t make developer’s job to try IE9 out any easier by not including an address bar. What kind of idiotic thinking is this? And when all I wanted to report a bug, they force me to sign up for a Live account before doing that. Well, forget it, too much hassle.
Per Dennis’s comment:
“Have you tried using one image, and then copying the image data beyond the first 8192 bytes mark back to 0,0 for IE9?”
I went back and tweaked the x() function, which handles the extraction of the image data, and got it to work in IE9. This means that IE9 will be able to extract data no matter how big the image is. The trick is to shift the image as it gets rendered on to the canvas by 8192 pixels each time. Essentially we render the image in “chunks” of 8192 pixels. So instead of splitting into multiple images using the build script, we can just use 1 image and let Canvas do the chunking for us. This is made possible because the encoded image is only 1px wide, hence we only need to shift the top position of the image and run getImageData() from 0 to the smaller value between chunkSize and the remaining height of the image. Here’s the updated function:
I wish I had thought of this work-around before hand, I could pack even more features into the 10K app.
