Alex Le’s Blog

Recently I had to configure my VPS at Slicehost.com to send out emails for my TubeCaption application. Due to the time constraint of the launch, we decided to outsource the mail delivery to Gmail SMTP server.

Using Gmail SMTP server with ActionMailer

When I first deployed the application, I didn’t configure sendmail correctly (I’m a total stranger to sendmail or postfix) and went ahead with Google’s SMPT server.  Since I am using Google Apps to host the mailboxes, I created an extra account specifically to use for mail delivery. In my development.rb and production.rb file, I have ActionMailer configured as follow:

ActionMailer::Base.delivery_method = :smtp
ActionMailer::Base.smtp_settings = {
:tls => true,
:address => “smtp.gmail.com”,
:port => 587,
:domain => “tubecaption.com”,
:authentication => :plain,
:user_name => “sample-mailer-account@tubecaption.com”,
:password => “the-very-secured-password”
}

Within my ActionMailer handler, I have a function to setup the default params as follow

class Postoffice < ActionMailer::Base
def setup_default_params
@from = “my-email@tubecaption.com”
@headers = “Reply-to my-email@tubecaption.com”
@subject = “TubeCaption.com”
@sent_on = Time.now
@content_type = “text/html”
end
end

Then for each email, before sending, I invoke setup_default_params() to have all the params setup. The site could now sending out emails, albeit slow. ActionMailer had to open a connection to GMail, authenticated, and then started writing the content as a stream of text. The whole process took from 1 to 3, or even 4 seconds some time on my machines, which means the mongrel process was also stuck there waiting.  But the mailing piece worked. However, I came to know the real issue:  all the emails were marked as SPAM or JUNK if they are sent to GMail or Yahoo.

I talked to a friend of mine, who has more knowledge in the emailing world. He suggested the cause of this is because emails sent from my VPS failed the reversed DNS lookup verification. Yahoo mail servers perform a reverse DNS look up to check my domain’s DNS records, while the SMTP server I used was from Gmail. Hence the emails were marked as spam in Yahoo and junk in Gmail due to the DNS check mis-match.

So I decided that it was about time for me to configure sendmail for the box and switch ActionMailer to use sendmail instead.

ActionMailer with local sendmail

Truthfully speaking, I would love to get away as far as I can from configuring sendmail. The 4th edition O’Reilly book on Sendmail is more than 1,000 pages! I only want something that can send out emails… Luckily, there is yum.

# sudo yum install sendmail

Now I switch my ActionMailer config back to use sendmail. Since my sendmail is installed in the default folder, I don’t need to configure ActionMailer any further. In production.rb, I have

ActionMailer::Base.delivery_method = :sendmail

ActionMailer now delivered mails using the local sendmail program.

Using the console, I sent out a test email to my yahoo acount.   Everything works.  Then, I received a notice from my server, telling me I have new mails at /var/spool/mail. Great! I checked the mail log, and saw …

—– The following addresses had permanent fatal errors —–
<nworld3d@yahoo.com>
(reason: 553 Mail from XX.XXX.XXX.XXX not allowed - 5.7.1 [BL23] Connections not accepted from IP addresses on Spamhaus XBL; see http://postmaster.yahoo.com/550-bl23.html [550])

—– Transcript of session follows —–
… while talking to b.mx.mail.yahoo.com.:
<<< 553 Mail from XX.XXX.XXX.XXX not allowed - 5.7.1 [BL23] Connections not accepted from IP addresses on Spamhaus XBL; see http://postmaster.yahoo.com/550-bl23.html [550]
… while talking to d.mx.mail.yahoo.com.:
<<< 553 Mail from XX.XXX.XXX.XXXnot allowed - 5.7.1 [BL23] Connections not accepted from IP addresses on Spamhaus XBL; see http://postmaster.yahoo.com/550-bl23.html [550]
…….
<<< 553 Mail from XX.XXX.XXX.XXX not allowed - 5.7.1 [BL23] Connections not accepted from IP addresses on Spamhaus XBL; see http://postmaster.yahoo.com/550-bl23.html [550]
554 5.0.0 Service unavailable

…….

Final-Recipient: RFC822; nworld3d@yahoo.com
Action: failed
Status: 5.5.0
Diagnostic-Code: SMTP; 553 Mail from XX.XXX.XXX.XXX not allowed - 5.7.1 [BL23] Connections not accepted from IP addresses on Spamhaus XBL; see http://postmaster.yahoo.com/550-bl23.html [550]
Last-Attempt-Date: Tue, 3 Jun 2008 14:02:54 -0400

OUCH! My VPS got black-listed! Somehow the Spamhaus XBL list decided to black list the IP address of the server. Probably because I had port 25 opened in the firewall and somebody has taken advantage of my lack-of-linux-sysadmin-skills to start relaying spams.  CRAP! Yahoo outrightly refused any SMTP connection from my server because it was marked as a spammer-wannabe .  Good thing Yahoo included the  link to their help page and references to Spamhaus XBL.

I went to Spamhaus website and looked up my server’s IP:

The CBL (composite blocking list) was blocking my server.  I went ahead and request a removal from the list.  Afterwards, the CBL list showed

A few minutes later, I checked Spamhaus and it’s not showing the IP listed anymore.  Phew!

It would take a while before this list is propagated through out the internet.  At the moment, I still can’t connect to Yahoo’s SMTP server and my mails is still delivered right into Google’s Junk mailbox.  Hopefully a few hours more and my server would be good to go.

I don’t know if Spamhaus is my friend or foe.  I have a hard time deciding it.  I don’t know what people’s experiences with emailing services in general, but I can tell it’s a pain in the butt, and everything will only get worse from here.  I wonder what it will be like when ipv6 is used…

I was implementing cache_fu for my upcoming application based on YouTube (will be premiered very soon here on my blog). I wanted to cache certain calls to the YouTube API to reduce the latency and the time mongrels got stuck waiting for the response. The first candidate to get cached was the “Top Rated Video” from YouTube. Since this section is a part of the homepage, I used fragment caching for it. Everything went fine, Memcached just ran so beautifully and the speed gain was exceptional. I added the :ttl option to the cache command to expire the contents, then suddenly nothing worked. Rails kept on whining for the cache helper call:

wrong number of arguments (2 for 1)

I checked and double-checked cache_fu source, and the fragment_cache.rb file in particular. I saw cache_fu extending the fragment cache helper with an extra hash argument to support the :ttl option. Why the hell I could not use it in my view?

It turned out that the fragment_cache overwrite in cache_fu was not called automatically. I had to manually added the call in environment.rb file to invoke the setup.  Strange! It took me a while to figure out since I traced all source files in cache_fu to find the bug. I have to admit, cache_fu (previously known as acts_as_cached) has some kick-ass Ruby code written. I still need to learn a lot more about Ruby to get to that level of code craftsmanship.

To fix this issue, include this one line in your environment.rb file

ActsAsCached::FragmentCache.setup!

That will ensure the cache helper to have the :ttl option.

It’s time to say NO to monopoly and for Net Neutrality to be passed as a law.

It’s not Comcastic as you all claim. It’s ComCraptic. My experiences
with Comcast has got from worse to worser, and now I am a very unhappy
customer. Comcast is in fact using dirty and shady tactics, often
disguised as to help “preserve the speed for its users”, to severely
throttle services that are not in its interest to run on the network.
YouTube doesn’t even want to stream sometime, meanwhile Hulu.com (a new
TV-show site) is streaming Hi-Def videos just fine and it is extremely
fast (because Hulu is not that big yet and so it’s not on the black
slow-list). Netflix Watchnow is now extremely slow and almost
unbearable. I watched a 24-min animation episode and it took 20 minutes
to download the video! I always have DUMeter installed so I constantly
monitored my download/upload rate. Before, for Netflix, I was streaming
at 500KBps - it was almost instant to begin watching the instant
videos. This is how the new “internet experience” should be. I was a
very happy Comcast customer.

I started to read on Slashdot about
Comcast trying to slow down BitTorrent and other traffic. NO! All
traffic on the Web should be considered EQUAL. Why are we trying so
hard to erase racial discrimination while Comcast is trying its best to
discriminate and destroy the foundation of “free internet” ? And NOBODY
right now can do a thing to change this evil Comcrap piece of @#$? What
would happen if Comcast begins to extort money from Netflix, YouTube,
Hulu, or even Amazon, to allow users to use these excellent services?
That would be the end of the free internet.

After a “service outage” in my area, the speed of Comcast degraded
worst than anything. YouTube streaming speed is out right ridiculous,
if not pathetic. Netflix streaming speed degraded from 500KBps to
300KBps, and now, as I’m typing this, it’s only downloading at much
less than 100KBps. It took 20 minutes to half an hour for anything to
work. This is pathetic. And Comcast did NOT disclose anything about
their practices, not until people started to discover concrete proofs.

(http://netflixcommunity.ning.com)

I found this diagram while searching on Comcast and its shady and
disrespects of the customer practice. I don’t think any explanation is
needed.

What kills the internet? Not spam, not phishing, not videos
streaming, not the data tsunami. It is the unfair practices from ISP
that slow down the evolution of the internet. Comcast is playing its
monopoly hand to dominate places where users don’t have an alternative.
And yet, they have all the power to fudge, deny, or change the TOS for
their own legal and financial benefits. I would pay double for my
internet connection if I have any choice other than Comcrap. That’s
right, I would pay $118 a month for any ISP that works and treats their
customers honestly.

In the US, everything is built on Trust, and Credit. Hence we have
credit history, credit cards, personal loans, and mortgages. If
corporations like Comcast are abusing and destroying this circle of
trust and credibility, society as a whole, will be mubh better off
without them.

Comcast, I sincerely wish you would go to hell for your practices.
Meanwhile, you will go to sleep better if you treat your customers with
dignity.

I’m in the process of installing Ruby on Rails on a VPS running CentOS 5.1 and run into an issue with gem complaining for the missing zlib library. What happened was that I first used “sudo yum install ruby” to install the ruby runtime. However, the pre-packaged ruby in the Yum repository was older, 1.8.5. I only found out about this when I tried to run the mongrel_cluster to set up the mongrel instances. Mongrel was complaining about the older version of Ruby and the need for the fcgi fix. I decided to uninstall the yum ruby package and compile the newer version of Ruby from the source.

I compiled and installed Ruby 1.8.6 from the tar ball source first (./configure, make, make install), then I removed the pre-installed package using “sudo yum remove ruby” and removed all the dependencies with it. Next I installed gem 1.1.1 from the source (ruby setup.rb). I proceeded to install rails 2.0.2 again (”sudo gem install rails -y”). Bam, gem started to complain:

/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`gem_original_require': no such file to load -- zlib (LoadError)

Not good!

I did some research and it turned out that my CentOS 5.1 VPS is a barebone box, it did have the extra development libraries installed. Hence when I compiled Ruby from source, make did not compile the zlib extension because I did not have the zlib-devel package installed.

After installing zlib-devel package through yum (”sudo yum install zlib-devel”), I cd’ed into my Ruby source folder and into the ext/zlib folder (in my case, ruby-1.8.6-p114/ext/zlib/). Then I proceeded to install the zlib library (”ruby extconf.rb”, “make”, “make install”). Afterwards, gem was working fine again! Yay!