Put your message here! Contact me for more information
 
 







 

Archive for the ‘Web Technolgy’ Category


 

I’m setting up a Rightscale server on EC2 running CentOS 5 to use RVM, however, I ran into the issue of the ca-bundle.crt that comes with curl is too old. The RVM installer script keeps on complaining about invalid certificates.


error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
More details here: http://curl.haxx.se/docs/sslcerts.html

Searching the net until now (2012!) still, sadly, doesn’t give that much results. So if you have SSL certificate issue with RVM, here’s your fix:


cp /etc/pki/tls/certs/ca-bundle.crt ~/ca-bundle.crt.old
sudo curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt

Basically the cacert.pem file provided by curl.haxx.se contains all the root certificates necessary for curl to verify Github’s SSL, and thus your RVM installer script will cruise happily.

view comments
 

While installing a new Ubuntu server on Linode for a client using the “Stackscripts for Ruby 1.9.2, Passenger, Nginx, Rails, and MySQL”, I ran into an issue with bundler unable to install a few gems, notably mislav/will_paginate and collectiveidea/delayed_job. The gemspecs files of these particular gems have UTF-8 characters, and rubygems barks with the following errors:

Using will_paginate (3.0.pre3) from https://github.com/mislav/will_paginate.git (at rails3) /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb:724:in `gsub': invalid byte sequence in US-ASCII (ArgumentError)

It turned out that for the default LANG environment variable wasn’t set by default. If you run

$ locale

you’d see something similar to this

LANG=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Notice that the $LANG variable is empty. Since ruby relies on the LANG to pick out the correct encoding, it got confused when trying to process the gemspecs containing the UTF-8 characters. To fix this issue, you can set the LANG option system-wise by adding 1 more line to your /etc/environment file. Just run this 1-liner and you’ll be all set.

$ sudo echo LANG=en_US.UTF-8 >> /etc/environment

Now the LANG will be set for your system, and thus will be passed to your ruby and passenger environment properly.

Here’s the updated version of the stackscript: http://www.linode.com/stackscripts/view/?StackScriptID=2166

view comments
 

I recently purchased and installed a new SSL certificate from GoDaddy for Marrily. During the process, I came to learn more about SSL and the different steps to set everything up from scratch. There are an abundant amount of articles and tutorials on how you can get started, but surprisingly there are no articles on “why” you have to follow those steps. Truth is I was pretty confused when I first started. There were a bunch of different steps and different key, pem, crt, csr files that need to be generated. The result was that I got lost and screwed up during the process. I then add insults to injury by accidentally revoking my certificate instead of re-keying it and ended up having to call GoDaddy to revert the deletion. Since any entrepreneur with a SaaS website will eventually need to implement SSL to protect their customers, having a better understanding of SSL will be greatly beneficial. This is my explanation to the entire process in plain English in hope that I can help clear up the confusion.

Why SSL?

To protect the communications between your web server and the client’s browser, you need to implement an encrypted channel so that all data transferred back and forth can only be read by your server and the browser. Anyone who eavesdrops in between will just see gibberish. Only your web server and the client’s browser know the right “secrets” to unlock the encrypted message. This communication protocol is called https, with the s stands for “secured”.

When user requests a page via https, your server will need to encrypt the content using a secret which the user’s browser can decrypt using a well-known identity. If somehow the content is encrypted with an unknown identity, the browser will be very hesitant to accept it, and it will ask user to make the hard decision to proceed or not.

Why Purchase a SSL Certificate?

To purchase a SSL certificate is to obtain a publicly verifiable identity for your domain that is accepted in all browsers. Most modern browsers include a list of well-known root Certificate Authority (CA) public keys, and any encryption done using these CA sources will be accepted by the browser. It is also possible for you to generate a root Certificate Authority set of key as well, technically speaking you become your own Certificate Authority. However, since your identity is unknown and not verifiable, the browser will not trust your keys and thus it will pop up an alert to notify the user. Nonetheless, once you add your certificate key to your browser’s list of accepted certificates, it will come to know about your identity and hence it won’t bother popping up anymore.

Since you can’t ask everyone to manually install your public key to their browser’s list of accepted certificates, you will need to buy the certificate from an established vendor whose public key already came bundled by default in the browser. I read somewhere that this is how browser vendors can make some money, e.g. the SSL guys will need to pay to have their identity (the public key) included in the browser. In exchange, these SSL vendors can turn around and certify (or “sign”) anyone who wants to get a SSL certificate for a fee.

If you think about becoming a SSL vendor, you will need to convince all other browsers that you’re completely trustworthy, and you protect your private key used to generate the SSL certificate with your life, since whoever gets their hands on your private key will be able to sign any SSL request, thus compromising your identity as the reputable Certificate Authority. All SSL vendors offer a warranty on their SSL certificate service from $1,000 to $10,000 to a lot more specifically as a statement that they keep their secret hidden really well to protect the identity of their customers’ SSL certificates.

Obtaining a SSL Certificate

Step 1: Generate your private key

To handle https requests, your web server will need to encrypt the data. Hence the first step you need to do is to generate a private key that will be used for the encryption. You can use different encryption algorithms but a SSL vendor can ask you to use a specific method and key length. The longer the key, the better the encryption strength. If the key is too short, the bad guy can quickly run through all the possibilities and found out your private key, then he can pretend to be you. In my case, GoDaddy want to have 2048 bits (256 bytes) for the strength for the private key. For personal use, a key strenght of 1024 bits (128 bytes) would be sufficient.

openssl genrsa -out private.key 2048
Generating RSA private key, 2048 bit long modulus
..............................+++
.+++
e is 65537 (0x10001)

Step 2: Generate a new SSL Request .csr file

The next step is to generate a “request” for a new SSL using your private key. This request file has an extension of .csr which stands for Certificate Signing Request, and it contains the identity about you (or your company), and most importantly, where the SSL certificate would be valid for: a single domain (cheapest) or any sub-domains (a.k.a. wildcard, and a bit more pricey). All these information will be encrypted using your private key and saved to a file. The SSL Vendor will then take this file and sign it to produce a valid SSL certificate that can be applied to your server.

EV SSL
If you pay more money, you can also get your identity in the SSL certificate confirmed as a legitimate business entity. This type of SSL certificate is called EV SSL (Extended Validated Certificate). Essentially the SSL vendor will verify the identity of your company by asking you to submit your business registration paperwork, bank account, letter from attorney or accountant, etc., for an additional fee ($400 to $1,000). In return, you will have a green-bar status with your company’s name next to the browser’s address bar. The theory is that user can identify your company’s name, and thus feels more secured as he/she knows that the website is the correct one, not a phished site that just pretend to be your website. Most (if not all) banks and prominent businesses have this type of EV certificate to protect their identity.

To generate a new CSR from your private key, use the command:

$ openssl req -new -key private.key -out marrily.com.csr

As I mentioned, the most important bit of the CSR file is where the SSL Cert should be valid for, which is defined in the “Common Name” attribute. For single domain (https://marrily.com, or https://www.marrily.com), you can use either “domain.com” or “www.domain.com”, since the “www” subdomain is so commonly used and thus can be omitted. Check out line 14 below for more details:

$ openssl req -new -key private.key -out marrily.com.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Marrily
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:marrily.com
Email Address []:alexle@marrily.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

I did not specify any challenge password in this case to keep everything simple.

Step 3: Submit your CSR to get a SSL Cert

Now that you have the CSR file containing your identity and which domain the SSL would be valid for, you can submit this CSR file to the SSL vendor (of course you will have to pay them first). They will take your CSR file and generate a new .crt (certificate) file using their own private key. Essentially they “sign” your CSR file with their carefully guarded secret file. You will then get back the your .crt file corresponding to the CSR, and another .crt file that belongs to the SSL vendor.

Chances that the SSL Vendor’s crt file actually contains a list of different certificates (public keys). The reason is that more or less your SSL vendor is actually a re-seller of another Certificate Authority, which can also be a reseller of another higher-level CA. So the first certificate would belong to your immediate SSL vendor, the one after that belongs to the higher-level CA that signed your vendor’s cert, and the cert listed after that belongs to an even higher CA that signed the CA’s cert that signed your vendor’s cert which signed your own certificate. Essentially it’s a tree of certificates that lead all the way up to the highest level of CA, which is a root certificate that is included in the browsers by default. For GoDaddy, the root CA is www.valicert.com, and for VeriSign, it is VeriSign’s own Class 3 Public Primay Certification Authority - G5.


(notice the green bar, that’s the EV SSL which costs you some more money to obtain)

Step 4: Configure Your Web Server

Now you should have in your possession these files:

1) your private key
2) your .csr file (not used anymore)
3) your new SSL certificate provided by your vendor as a .crt file, which is valid for your domain.
4) your SSL vendor’s crt file, containing a list of different certificates.

You are now ready to go and configure the web server to use your private key and your new SSL certificate (which is technically a public key) for the https-enabled website. The specific configuration for each web server is different, but the process will be the same. Also, the .crt files sometimes have a “.pem” extension as well, but for simplicity’s sake, they can be used interchangeably.

Nginx and GoDaddy SSL

In my case, I used nginx to serve my Rails application. I originally installed this nginx instance from source using passenger’s installer but ssl was not enabled by default (you can check this by running “nginx -V” and look for - -with-http_ssl_module). I re-ran the passenger’s installer again and add the - -with-http_ssl_module switch to the optional parameters, and everything was good to go.

One gotcha for Nginx is that you will have to combine the 2 certs that GoDaddy give you into one .crt file, with your SSL certificate comes first, then GoDaddy’s crt file (gd_bundle.crt). The browser would understand this as your SSL was signed by the CA whose public key is next cert entry, then that one was signed by the one after it, etc. all the way to the root CA.


$ cat www.marrily.com.crt gd_bundle.crt > marrily_combined.crt

I then added a new server{} block to listen for ssl requests on port 443. After restarting Nginx, Marrily is now ssl-protected with a green padlock.

server {
    listen          443;
    server_name     marrily.com;
    # passenger stuff

    ssl on;
    ssl_certificate         /your/ssl/folder/marrily_combined.crt;
    ssl_certificate_key     /your/ssl/folder/private.key;
}

Self-Signing your Certificate and Testing SSL Locally

Now that Marrily is https-enabled and some of the actions requires SSL, I wanted to develop the site locally using SSL as well to make sure all the logic worked correctly. I’d need to self-sign a new SSL certificate and have it installed locally.

Preparation
In my environment (Mac OS X Snow Leopard), I also have nginx installed using Homebrew. Homebrew installed nginx with ssl support by default so no recompilation was needed. I also added a new entry to my host file so that I can use a fake domain to access my local site, and I’d use this fake domain in my CSR as well.

# /etc/hosts
127.0.0.1 marrilydev.com

Self-Signing a New Certificate
I generated a new private key using openssl:

$ openssl genrsa -out privatekey.pem 2048

Then I generated a CA cert using this private key:

$ openssl req -new -x509 -key privatekey.pem -out cacert.pem -days 3650
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:marrilydev.com
Email Address []:

I didn’t care about any of the details except for the Common Name field, which I specified the fake domain.

Since the cacert.pem file was generated (a.k.a. signed) using the same privatekey.pem file, we could use it as the SSL certificate directly. All we’d need to do is set the ssl_certificate_key setting in the configuration to the privatekey.pem file:

upstream rails { server 127.0.0.1:3000; }

server {
   listen       443;
   server_name  marrilydev.com;

   ssl                  on;
   ssl_certificate      /Users/sr3d/projects/misc/ssl/cacert.pem;
   ssl_certificate_key  /Users/sr3d/projects/misc/ssl/privatekey.pem;
   ssl_session_timeout  5m;

   server_name   marrilydev.com;
   access_log    /Users/sr3d/projects/marrily/svn/marrily_marrily/m3/app/log/access.log;
   error_log     /Users/sr3d/projects/marrily/svn/marrily_marrily/m3/app/log/error.log;
   root          /Users/sr3d/projects/marrily/svn/marrily_marrily/m3/app/public/;

   location / {
     proxy_set_header  X-Real-IP  $remote_addr;
     proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header Host $http_host;
     proxy_connect_timeout 74; # max is 75s
     proxy_redirect off;

     # Proxy to Backend
     if (!-f $request_filename) {
        proxy_pass http://rails;
        break;
     }
   }
}

(Note: locally I have the nginx proxy all traffic to the development server running on port 3000)

Also, since Mac OS X has special restrictions for port 80 and port 443, nginx must run with sudo to listen to port 443, otherwise it would silently fail and you won’t be able to hit the site via https.

Getting Rid of SSL Warnings by Installing the Self-Signed Cert
With nginx configured to listen to secured requests, I opened up the site in Chrome, and saw a huge red error message complaining about the validity of the certificate, since Chrome did not recognize the identity of the cacert.pem. Obviously I could just ignore the warning and proceed to the https site for the current session, but there’s a better solution: add the cacert.pem to the list of approved certificates.

To install the self-signed certificate, just double click on the cacert.pem file in Finder. The cert would be added automatically to Keychain Access.

With the cert added to Keychain, all browsers installed in the system would gladly accept a https connection to https://marrilydev.com.

Summary

  • SSL certificate is not all that confusing once you understand the gist of it and why each file is needed
  • The process in simple steps:
    • generate a new private key for encryption
    • Using this private key, generate a CSR containing the domain information for the SSL
    • submit the CSR file to the SSL vendor to obtain a new CRT certificate file
    • configure your web server to listen to 443 https traffic using the private key in step 1 and the CRT obtained from the vendor
  • GoDaddy SSL has different pricing on their SSL stuff, so search around and don’t pay a full price.
  • SSL is cheap, implement it to protect your customers and gain their trust
  • If you’re gziping your site, should add this line to your nginx’s conf file:
    gzip_buffers 16 8k; to make sure nginx doesn’t loose large gzipped JS or CSS

Reference

view comments
 

I submitted my application entry, GithubFinder, to the 10K Apart Contest last night. GithubFinder is basically a Mac-like Finder app to explore a Github repository. A lot of the time when I’m visiting Github, I wanted to quickly browse a repository, but there isn’t a quick way to do so. The 10K Apart contest gave me a reason to build something cool that I also could use. The rules to the contest are: the app needs to be less than 10K in total and it has to be 100% client-side and 100% compatible with IE9, Firefox, and Chrome/Safari. That means everything, markup, images, stylesheets, and CSS, need to be less than 10 KB (10,240 bytes) and there won’t be any server-side processing for the app to work. Packing a high-impact application into 10,240 is surprisingly not simple, I’ve learned quite a few things working on GithubFinder.

I wanted to share a technique that was invented by NIHILOGIC. The idea is to store the javascript code inside a PNG file to leverage the image compression and significantly reduce the size. And this technique can also be applied to CSS as well. I hope that my implementation would benefit someone before the 10K contest end.

From the early stage of the app, I know that I’d want to go over the 10K limit (well, limits are there to be pushed, and rules are set to be broken!) I really wanted GithubFinder to be useful, even in a minimal form. Moreover, I’m not the briefest coder in the world, so I decided to cheat the 10K rule by using Nihilogic’s inventon. I had read his article a while back, and I was amazed by his 8KB Mario implementation. Since IE9 supports Canvas, now is the time to whip out this trick. However, coming back to the article, the script to encode the JavaScript was no longer available. I ended up spending a few hours to re-created the javascript-to-png script again in Ruby, and a long the way, wrote an automate build script for the application.

If there’s anything in this project I’m really proud of, probably it’s my build script. The build process for the application works like this: all JS and CSS files are merged and minified using YUICompressor, then the minified JS and CSS are concatenated into one string, separated by a unique delimiter. This long string is then encoded into a PNG image. If there is any error in the javascript, YUICompressor would prompt and I’d jump to the error line, fix, then re-run the build to verify. Here’s the first part of the build-script containing the utility methods to handle all the merging, compressing, and generating image.

The merging and minification are both done inline to not litter the application with temporary files. Using Open3, I was able to just use yuicompressor directly (line 19). The interesting bit is the string2png() and its sibling, and the string2pngs() method. Here’s the string2png() method again:

string2png would read the input string character by character and convert this character to its ASCII equivalent, then create a new pixel with the red color of the ASCII value (line4 and 5). Afterward, we compress the image using the ZipCompression and write it out as an 8-bit PNG file. If you open up this file in Photoshop, you’ll see a long vertical stripe of different shade of red. One gotcha for this technique is that you won’t be able to use UTF-8 characters since we don’t have ASCII equivalents for them.

On the front-end, I created a “bootstrap.js” script similar to Nihilogic’s one to read back in the image file. Essentially the bootstrap.js script would load up the image, paint it on the canvas, then read back the pixel one-by-one and grab the red color out. Since the characters are stored in binary format and compressed, the code can greatly be minified.

The image produced by string2png works well in Chrome and Firefox, however, with IE9, there is a bug preventing the image to be read back properly. IE9’s getImageData() could only read the first 8192 bytes of the canvas pixel data, everything after the 8192th pixel are zero’ed out (you can see my test cases here — IE9 would consistently fail). (SEE UPDATE 1 below for the fix) The work around is to split the image into smaller ones, each has a maximum of 8192 pixels. The bootstrapper’s job now is to read these image files sequentially back in before reconstructing the js/css string. Hence we have string2pngs method:

The down side of this IE9 work-around is that the compression efficiency is reduced, since intead of having just 1 image, we’d have 2 or 3 images, and they all have to store their own color palette separately as apposed to sharing a single one. I estimate that I lost about 20% of storage space(1 single image: ~9.5KB, 3 equivalent images: ~12KB) due to the the splits. I ended removing features so that I could fit everything into the 10K limit.

At any rate, here’s the bootstrap.js script to handle the decoding of the image:

The function of interests here is x(). The whole decoding businesss are done from line 26 to 29, then the decoded string are passed back to the callback. Please bear with me the short variables, I was trying to minify this script. The run() call back would check for the chunks, and only proceed when it receives all the chunks. My earlier naive implementation of run() had a bug causing by a race condition which the images were re-assembled in different orders, thus sometimes the app initialized properly, and some other times it would fail.

Together with the build script, I had a quite flexible mechanism to tweak and play with the output files to get to the targeted size. Click here to see my full build script. I hope this help people participating in the 10K contest to create more wonderful applications.

Finally, my verdict on IE9 Platform Preview release? Unimpressive and buggy. I was able to crash it reliably (!). Microsoft should have waited a few more weeks to put in more features before rushing to release a piece of inferior software. Even worse, they didn’t make developer’s job to try IE9 out any easier by not including an address bar. What kind of idiotic thinking is this? And when all I wanted to report a bug, they force me to sign up for a Live account before doing that. Well, forget it, too much hassle.


UPDATE 1:

Per Dennis’s comment:

“Have you tried using one image, and then copying the image data beyond the first 8192 bytes mark back to 0,0 for IE9?”

I went back and tweaked the x() function, which handles the extraction of the image data, and got it to work in IE9. This means that IE9 will be able to extract data no matter how big the image is. The trick is to shift the image as it gets rendered on to the canvas by 8192 pixels each time. Essentially we render the image in “chunks” of 8192 pixels. So instead of splitting into multiple images using the build script, we can just use 1 image and let Canvas do the chunking for us. This is made possible because the encoded image is only 1px wide, hence we only need to shift the top position of the image and run getImageData() from 0 to the smaller value between chunkSize and the remaining height of the image. Here’s the updated function:

I wish I had thought of this work-around before hand, I could pack even more features into the 10K app.

view comments
 

I needed to update “../../lib/prototype.js” in all html files in the folder to “../../prototype.js”.  Since sed is installed by default in Mac OS X (I’m using Snow Leopard), this is a one-liner:

sed -i “” ’s/lib\/prototype/prototype/g’ *.html

Notice the “” after the -i params.  This is to replace the existing files with the updated files. Neat!

If you get “invalid command code” when you run sed, you probably miss the empty double-quote “” pair after the -i.  Sed on the Mac has slight different in terms of syntax so you need to have the empty double-quote pair.

view comments