Alex Le’s Blog

Today is quite a productive day for me, not for schoolwork but personal research on the Web 2.0 Stuff. I still have a semester-long Inorganic Lab notebook to finish and turn it in by the end of this week. I will be leaving soon to work on that but I cannot resist the temptation to type this little report up, especially after I just did a quick Ajax experiment on low-lag client/ server communication (see my previous post.) Ajax works but fails when it comes to low lagging. So how come Meebo, Gtalk, and Gabbly can do this ultra low-lag? What is the special technology/ platform are they running?

Gabbly
I went to Gabbly and at the time, there were 2 people who claimed to be from Gabbly: Jon & Abe. I asked them about what webserver Gabbly was running on and there was no answer from them. Well, this is understandable as Gabbly is their (not verifiable) product. Anyway, using FireBug (a FireFox extension), I was able to see the request made from Gabbly. The header tells the whole story:

They are using Apache-Coyotte 1.1 as their web server!

How Gabbly Works: Ajax & JSON
I think this is how everything works, the comet way: Basically the page constantly making request to the server. The connection is kept-alive (will be the buzz word soon!) for a x amount of seconds (30 secs?) After this period, the server flushes the data back to the client. The client receives the ajax response in JSON format, which most of the time is empty (firebug says that it’s “not well-formed”) as nothing is returned.

On the other hand, if there is any new data, a sample JSON object being flushed by the server is:

The client, after receiving the JSON, will get busy right away by making another request to the server, waiting for more contents to be “pushed”. Of course the ajax engine will be parsing the JSON object to display the message to the user. Yay, happy communication.

The Problem
The ultimate question is: how the server can handle multiple keep-alive connections? There’s an article describing this problem, a.k.a the C10K problem (http://www.kegel.com/c10k.html). C10K means the server has to serve 10,000 concurrent connections. Apache is said to quickly fail this task as the server keeps forking (making a copy of itself) to handle more connections. After awhile, apache can consume all the system resource: game over. Obviously, for Comet to work, a new type of web server needs to be developed. Luckily, Gabbly has proved there is indeed a solution by providing a working product. But they are running Apache-Coyotte 1.1 as the header says. Truthfully speaking, I have no idea about Tomcat/JSP. I do know Java and J# but using Java for web, in my opinion, is an overkill and anti-cost effective for small web developers.

The new million-dollar question is: how the rest of the world, where PHP is the champion, can use Comet?

Meebo
I think Meebo is THE example of Comet applications. It’s fast and a pleasure to use. It can very much be used as an alternative to a desktop IM application. Quickly comes the C10K problem again. But again, if there is no viable and sound technology fundamental, Meebo won’t be able to exist and got funding from the california venture capitalist firms (I do monitor the VC news closely to see the trend of the web!) I did a little investigation using Firebug and I found the multi-dollar answer to our question.

The (hopefully) million-dollar answer: Lighttpd comes to the rescue. (And later on we expect to see Apache with its mod_mpm too.)

Ah hah! Lighttpd comes to the rescue! I quickly went to Lighttpd website and looked up the Features List: Lighttpd is indeed built for high concurrency. And more importantly, Lighttpd can fully support PHP. YAY for the opensource community and for me myself. I’ve been having an idea about a project that heavily involves Ajax and Comet. If I can use Lighttpd for Comet, then I can use PHP, and I am confident that my project is viable as I now have the fundamental technology to support my idea.

Again, we see the familiar JSON object from the Meebo server response. And similar to how Gabbly works, the web browser open up a connection right away to wait for new data to be pushed.

That’s it…
Well, that’s it for today. I have to go and start on my Inorganic Lab notebook. 2 more days till freedom. 2 more weeks to graduation and I’ll be done for good. I hope that I did answer your (and my) question about how Gabbly and Meebo work and the support technology behind.

PS: Comments are very welcome!

I am doing a little experiment on using Ajax for moving an object on the screen. My ultimate goal is to sync a position of an object on the screen with the server. So this experient serves as a test to examine the speed between the server and client, also the bandwidth used to keep up the ajax request. What I found with my baby test is that with my current technique, moving an object asynchronously with server is NOT fast enough for a smooth experience. Yes it can be done but it is way too slow.

The setup:
A simple div with absolute position. I used Prototype for Ajax request.

The client:
Arrow buttons input from keyboard is bound into a function and an ajax request is sent to server with the TOP and LEFT coordinates of the target object and the direction of movement.

The server:
With the TOP and LEFT data from POST and the direction, I compute the new coordinate by add/ subtract the direction with a predefined speed. For now, I wrote a little PHP script to accomodate this feature on the server. After I have the new coordinates, I simply echo back and the client’s prototype library will take care of the rest.

What I found
In my localhost, the request is fast enough witouth any lag. On 1and1 server, where this blog is also hosted, the ajax move experiment becomes extremely jerky and laggy. The round trip request has taken its toll. There’s about roughly 0.5 to 1 second lag between each keystroke especially if I keep pressing an arrow key to make continuous requests. The bandwidth used is about 2 to 3 Kbyte/s for continuous round trip ajax request

Conclusion
Ajax can be used to request info dynamically from the server, but it is not suitable for “streaming” data or quick communication (low lag) between client/ server. A different communication method has be used. I highly suspect that COMET is capable of this type of client/server communication but I don’t know how to test this method yet.

Demo
AjaxMove demo can be found here. The code is tested to work with Firefox.
http://alexle.net/experiments/ajaxMove/test.php

I just spent my entire afternoon and most of my evening today working on this little “google maps” clone. Basically I created a map linking my kingdom (a.k.a. my blog) to my projects and my friend’s website. The code was largely based on the Pragmatic Ajax book with my own modifications (keyboard attachment, moving the viewport to a certain tile) I think it’s pretty cool to see how the Google Maps has been done. Now I can have my own version of my world map too.

The link is here if you want to go directly to it. Otherwise, check out the iframe window below. The code is quite buggy at this time because this is purely javascript hacking.

If you open the page in its own windows, you can use the arrow buttons to navigate or the menu for quick jump (I don’t think Iframe can properly catch the event trigger to enable keyboard scrolling)

Well, 999 hits and counting over extensions.joomla.org. This is quite a success for me considering the work that I put in(not much actually, :wink;)

I am planning on improving the javascript part of EiP to use createElement instead of the innerHTML thing as I tested on FF1.0x, the component somehow fails. I’m also thinking about a edit-in-place on-steroid version: instead of 1 form element at a time, how about being able to update the entire form in one click? The developer can only need to make a div to wrap around, then do some javascript initialization and everything is good to go. Why am I doing this? Well, writing a web-app is tedious sometime. You have to develop the front-end, the the backend with additional adding/updating features. So the work is trippled. With this on-steroid ajax form, the developer can just write the frontend and easily add the updating features to it.

If you have used and happen to like Edit in Place, let me know what you think.

I had a really good surprise today in my class, Cryptography. Prof. Yuen (amazing guy, extremely smart) invited an old guy who brought with him an Enigma Machine. This guy had actually borrowed the machine from a friend of his. He said that his friend bought the enigma machine for $25,000 a few years ago and it had been sitting in the storage for years before he borrowed it to demo for previous Crypto classes. Considering the small 25-pound wooden box and little rotors inside, 25 grands is quite a bit of money (I could have fund 1 and 1/2 yesr of college with that money!)

The machine was in very good condition. The rotors were working fine, the keyboard was in amazing condition. We didn’t test the light bulbs since the machine used a 4-volt battery. So the guy took apart the rotors and passed it arond the class. Those rotors must weigh at least 1 pound or no less. Suppossingly the circle on the rotor could be twisted but since the machine had been in storage for at least a year, the circle was stuck. After the presentation, we were allowed to play with the instrument for a bit (which is awesome!) This is probably once-in-a-lifetime change as who’d let anyone played on an antique item that cost at least 40 grand? Anyway, the keyboard was pretty heavy to type, imagine a crappy typewriter with keys that take one pound for each stroke. The rotor did rotated every stroke and the next rotor rotated every 26 strokes of the first stroke to facilitate the permutation of the characters. Hey, this is really an electro-mechanic encryption machine and it’s ubber cool!

Compared to the current standard of keysize, the Enigma keyspace for the 3-rotors is pretty lame. But it was some 60 or 80 years ago when hundreds of lives were actually dependent on the little box, it had some significant meanings to it. Now the enigma machine is no more enigmatic, it costs a fortune to have one of those babies. Incedentally, at the time of the presentation, a guy on Ebay was selling one similar machine. The bidding was around 24k but still “reserved-not-met”. The machine actually can easly worth 40 grands if not more.

Thanks to Prof. Yuen and his friend for letting me have this chance to experience the historic machine. It’s a blast! Last thing, I’d highly recommend taking a Cryptography course for anyone who’s interested in Computer Science. This is where real science is combined with real application. When does someone have a chance to study about DES, 3DES, AES, RSA, Public Key Exchange, Eliptical Curves? College! I’m blessed to be able to do it. But again, I cannot wait to graduate. 2 more days to the last day of my college career. 2 more weeks to commencement. See you there.