Alex Le’s Blog

I ran into an issue with latest Devise (1.2.0 6e71eca).

BCrypt::Errors::InvalidSalt in SessionsController#create
invalid salt


This was caused due to the fact that in later versions of Devise, the default “cryptor” was changed to :bcrypt instead of :sha1. I did not specify the default encryptor before, and thus the issue with encryption algorithms mismatch arises. This means that all the existing users created using older version of Devise would not work with the new Devise after running “bundle update”.

To fix this issue, in the devise.rb initializer file (config/initializers/devise.rb), specify the encryptor to :sha1 (the previous default).

config.encryptor = :sha1


I was caught off-guard with this issue since all my test cases passed with flying colors. For Marrily I use FactoryGirl and Shoulda to test. In the setup method of most test cases, a new user object is created from a factory definition. Since the generated password and salt of the test user are always created with the latest code from Devise, this issue of cryptor mis-match will never happen. Had I used static fixtures, this bug would be caught, because the salt would always be static and thus Devise would throw an exception. In an existing database with users generated from a previous version of Devise, the change could be a huge problem. To fix the issue, all registered users after the Devise gem upgrade will need to have their passwords reset with the right cryptor.

Marrily has started out with Rails 3beta2 and it’s been upgraded ever since all the way to Rails 3.0.0. I launched the closed-alpha version August 1st, and had many users signed up using the :sha1 cryptor. Had I released the latest code to production without specifying the cryptor, it would probably cause a lot of issues to the site. The lesson learned here is to make sure that you are as specific as possible with your plugins, especially for default values. When in doubt, specify it. And really test your application. There’s no way around it. Rails and many popular plugins and gems move fast. To protect your application, having a test suite is the only way to ensure a smooth sail.

This summer I’ve been reading quite a bit. I read about Richard Feynman and found out that he is an original and very inspirational person. His outlook on life was extremely positive, and he was one good story teller. I realized that many successful people are in fact good story tellers. Good story telling is not just about communications, it’s about putting the listeners into the scene to engage them, to help them visualize the settings and to imagine the reality of that story.

I have never been very good at telling people what’s in my head, or what happened to me. Most of the time, the other person lost track of what I was saying after a while. I felt frustrated, shut up, and moved on, but deep inside I know that something is missing in the way I’m communicating. So I did a Google search on “how to be good at telling story” and got 13.8M results. Apparently many people are wondering about this too. The number 1 result, DumbLittleMan’s “A Formula for Telling a God Story” actually has very good tips on this topic.

The formula is broken down into 6 parts: Who, What, Where, Conflict, Resolution, and Tag. The story needs to tell the listener the person (who) involved in the situation (what) and the setting of that situation (where). Then the main point of the story is the conflict part, why this story matters. Then finally, the resolution of this conflict and a conclusion (tag), witty if possible.

My weakness is in the building up an arch of story that the listener can relate to. It’s just like hearing one’s own voice on a video for the first time and realizes how different it sounds than the way one hears it. As the person who’ve experienced the story, I have all the background information on why this bit of story is interesting, however, the other person doesn’t. I ended skipping too many details to help him/her fill in the blank. Like a puzzle that missing so many pieces that the picture is spotty with holes.

Another weakness of mine is in the way I explained the conflicts. I’d get excited and went too fast, and along the way, the listener was lost. I could detect the slightest change in the listener’s body language that he/she wasn’t engaged in the story anymore, and I tried to explained more, but only to confused that person further. I’ve been telling myself to slowdown a bit so that my brain can work out more details, and reduces the use of filling words such as “uh”, “uhm”, “like.” My goal is to stop using these words and substitue them with small pauses instead.

The DumbLittleMane’s article is a good start for me, and like all things, practice makes perfect. I’ll revisit this topic later on when I detect any improvements.

I submitted my application entry, GithubFinder, to the 10K Apart Contest last night. GithubFinder is basically a Mac-like Finder app to explore a Github repository. A lot of the time when I’m visiting Github, I wanted to quickly browse a repository, but there isn’t a quick way to do so. The 10K Apart contest gave me a reason to build something cool that I also could use. The rules to the contest are: the app needs to be less than 10K in total and it has to be 100% client-side and 100% compatible with IE9, Firefox, and Chrome/Safari. That means everything, markup, images, stylesheets, and CSS, need to be less than 10 KB (10,240 bytes) and there won’t be any server-side processing for the app to work. Packing a high-impact application into 10,240 is surprisingly not simple, I’ve learned quite a few things working on GithubFinder.

I wanted to share a technique that was invented by NIHILOGIC. The idea is to store the javascript code inside a PNG file to leverage the image compression and significantly reduce the size. And this technique can also be applied to CSS as well. I hope that my implementation would benefit someone before the 10K contest end.

From the early stage of the app, I know that I’d want to go over the 10K limit (well, limits are there to be pushed, and rules are set to be broken!) I really wanted GithubFinder to be useful, even in a minimal form. Moreover, I’m not the briefest coder in the world, so I decided to cheat the 10K rule by using Nihilogic’s inventon. I had read his article a while back, and I was amazed by his 8KB Mario implementation. Since IE9 supports Canvas, now is the time to whip out this trick. However, coming back to the article, the script to encode the JavaScript was no longer available. I ended up spending a few hours to re-created the javascript-to-png script again in Ruby, and a long the way, wrote an automate build script for the application.

If there’s anything in this project I’m really proud of, probably it’s my build script. The build process for the application works like this: all JS and CSS files are merged and minified using YUICompressor, then the minified JS and CSS are concatenated into one string, separated by a unique delimiter. This long string is then encoded into a PNG image. If there is any error in the javascript, YUICompressor would prompt and I’d jump to the error line, fix, then re-run the build to verify. Here’s the first part of the build-script containing the utility methods to handle all the merging, compressing, and generating image.

The merging and minification are both done inline to not litter the application with temporary files. Using Open3, I was able to just use yuicompressor directly (line 19). The interesting bit is the string2png() and its sibling, and the string2pngs() method. Here’s the string2png() method again:

string2png would read the input string character by character and convert this character to its ASCII equivalent, then create a new pixel with the red color of the ASCII value (line4 and 5). Afterward, we compress the image using the ZipCompression and write it out as an 8-bit PNG file. If you open up this file in Photoshop, you’ll see a long vertical stripe of different shade of red. One gotcha for this technique is that you won’t be able to use UTF-8 characters since we don’t have ASCII equivalents for them.

On the front-end, I created a “bootstrap.js” script similar to Nihilogic’s one to read back in the image file. Essentially the bootstrap.js script would load up the image, paint it on the canvas, then read back the pixel one-by-one and grab the red color out. Since the characters are stored in binary format and compressed, the code can greatly be minified.

The image produced by string2png works well in Chrome and Firefox, however, with IE9, there is a bug preventing the image to be read back properly. IE9’s getImageData() could only read the first 8192 bytes of the canvas pixel data, everything after the 8192th pixel are zero’ed out (you can see my test cases here — IE9 would consistently fail). (SEE UPDATE 1 below for the fix) The work around is to split the image into smaller ones, each has a maximum of 8192 pixels. The bootstrapper’s job now is to read these image files sequentially back in before reconstructing the js/css string. Hence we have string2pngs method:

The down side of this IE9 work-around is that the compression efficiency is reduced, since intead of having just 1 image, we’d have 2 or 3 images, and they all have to store their own color palette separately as apposed to sharing a single one. I estimate that I lost about 20% of storage space(1 single image: ~9.5KB, 3 equivalent images: ~12KB) due to the the splits. I ended removing features so that I could fit everything into the 10K limit.

At any rate, here’s the bootstrap.js script to handle the decoding of the image:

The function of interests here is x(). The whole decoding businesss are done from line 26 to 29, then the decoded string are passed back to the callback. Please bear with me the short variables, I was trying to minify this script. The run() call back would check for the chunks, and only proceed when it receives all the chunks. My earlier naive implementation of run() had a bug causing by a race condition which the images were re-assembled in different orders, thus sometimes the app initialized properly, and some other times it would fail.

Together with the build script, I had a quite flexible mechanism to tweak and play with the output files to get to the targeted size. Click here to see my full build script. I hope this help people participating in the 10K contest to create more wonderful applications.

Finally, my verdict on IE9 Platform Preview release? Unimpressive and buggy. I was able to crash it reliably (!). Microsoft should have waited a few more weeks to put in more features before rushing to release a piece of inferior software. Even worse, they didn’t make developer’s job to try IE9 out any easier by not including an address bar. What kind of idiotic thinking is this? And when all I wanted to report a bug, they force me to sign up for a Live account before doing that. Well, forget it, too much hassle.

UPDATE 1:

Per Dennis’s comment:

“Have you tried using one image, and then copying the image data beyond the first 8192 bytes mark back to 0,0 for IE9?”

I went back and tweaked the x() function, which handles the extraction of the image data, and got it to work in IE9. This means that IE9 will be able to extract data no matter how big the image is. The trick is to shift the image as it gets rendered on to the canvas by 8192 pixels each time. Essentially we render the image in “chunks” of 8192 pixels. So instead of splitting into multiple images using the build script, we can just use 1 image and let Canvas do the chunking for us. This is made possible because the encoded image is only 1px wide, hence we only need to shift the top position of the image and run getImageData() from 0 to the smaller value between chunkSize and the remaining height of the image. Here’s the updated function:

I wish I had thought of this work-around before hand, I could pack even more features into the 10K app.

I decided yesterday to work with VIM (MacVim) fulltime for a week and see how it turns out.  It’s been 24 hours, and so far my experience isn’t all that great!  I miss Textmate really badly.  And even though Textmate lacks certain features, working with Textmate isn’t all that bad.  So why change things that has already been working?

Textmate and its brother E-texteditor both have great success because they have features that tremendously improve files and text navigation, and things are just work if you do web development.  I rarely have to dig through my project finder to locate a file since Cmd T (or Ctrl Shift T in E) is there to save the day. Then Cmd Shift T (or Ctrl L foin r E) brings me directly to where I need to be.  Everything happens within a second or two second from knowing where I need to go to getting there.  With Bundles and Zen-Coding support, the editor becomes a deadly tool to slice through any kind of web development.

The biggest reason why I wanted to try Vim out is the split screen support.  I’m really spoiled by the wonderful E-texteditor, and I expect that my other tools would be as nice.  Textmate comes close, but stubbornly doesn’t have split screen.  Ironically E-Texteditor is supposed to be a Textmate clone for Windows, but it ended up with much, much better support for multi-file editing with great split-screen support.  While split screen in VIM works, it doesn’t feel as nice.  I may have not used VIM long enough (however, this is probably my 3rd or 4th attempts to use VIM), but I don’t want to bend backward to find myself trying to work with a text-editor.

Several plugins in VIM have tried with various success to bring Textmate features over, notably CommandT, BufferExplorer, NERDTree, Vim.Rails, and CTags.  Together they solve the issue with file navigations.  However, the other big missing piece is “Go to symbol” feature, and this is really critical for me:  I have to routinely work with CSS files, and without fuzzy searching the classes with CSS selector scope, it is a pain.  For example Textmate, to go to #wrapper .title, I’d need to only hit Cmd Shift T and type w r t i<ENTER> and be done.  In VIM, I have to use / to search for a match, but I don’t have the fuzzy scope select to quickly drill down to the class, so I’d need to further digging through.  I could make my css classes more specific to ease the searching in VIM, but I don’t want to.  My CSS file right now is more than 800 lines, but moving within it in Textmate is a breeze (I use one-liner for CSS class, e.g. selector and declarations are on the same line, #wrapper {… } so 800+ plus lines are 800+ css rules).  Until VIM has a plugin to solve this, I won’t be considering VIM for full-time use.

Personally I feel that my productivity in Textmate is not that bad, so why change things that is not broken?  Granted with VIM, it’s a really powerful editor, but being a pragmatic developer means to use whatever that works for you, I decide that Textmate is good enough.  VIM takes too much to customize to the level of functionalities that I want, and using it so far makes me unhappy.  If VIM is 100% powerful, but need 110% of customization and learning curve to master it that, then I’m happy with Textmate for 90% of the needed functionality and a much better and happier experience.

I finished “Surely You’re Joking, Mr. Feynman!” the other night, and one of Feynman’s living philosophies struck me:  ”so I decided it would always be chocolate ice cream, and never worried about it again.”  (An Offer You Must Refuse chapter)  So I’m going to adopt his way to decide that I will stick with Textmate, and work around it quirks with split-screen.  I admit that I have failed my own self-challenge, but in the end, I’m not too concerned about that.  I made a decision based on my own experience, and that will be it. Vim is forever, but Textmate is good enough for now.

I needed to update “../../lib/prototype.js” in all html files in the folder to “../../prototype.js”.  Since sed is installed by default in Mac OS X (I’m using Snow Leopard), this is a one-liner:

sed -i “” ’s/lib\/prototype/prototype/g’ *.html

Notice the “” after the -i params.  This is to replace the existing files with the updated files. Neat!

If you get “invalid command code” when you run sed, you probably miss the empty double-quote “” pair after the -i.  Sed on the Mac has slight different in terms of syntax so you need to have the empty double-quote pair.