Three Good Things: 12-06-2006
Published Thu, Dec 7 2006 1:43 AM
Technorati Tags: Computers and Internet, Software Development, Blogging, Annoyances
Code that works. Wednesday's Heroes Blogroll. The key to happiness.
I'm not really sure I understand spammers. Well, maybe I understand them better than I want to. It's been said that the love of money is the root of all evil. It's certainly true that it's the root of a lot of spam.
It's bad enough when spammers manage to get your email address. Roughly 30% of the email I receive is spam, and I receive a lot of email. It's also amazing what spammers think you'll be interested in. Something like 25% of the spam I receive is for pornographic web sites. Another 30% is for cheap pharmaceuticals "without a prescription". 32% is people trying to get me interested in buying junk stocks. Another 10% is from phishing sites. The latest variant of these comes to me from "E-Bay" purporting to be a dispute resolution claim over an item I'm supposed to have sold, but never delivered. Never mind the fact that I've never sold anything on E-Bay, and don't even have an E-Bay account. The rest is just plain garbage, random strings of words that almost, but never quite make sense, or it has malicious attachments.
I've tried a lot of anti-spam strategies over the years. Ten years ago, when spam was much rarer than it is today, I would dutifully download the messages, examine the headers, and send a copy along to the "abuse" alias at the spammer's ISP. Sometimes that wasn't so easy. I had to do IP address lookups to see who the ISP was since the spammer typically ran from a local SMTP server. Most of the time, nothing would come of it, but I did get the satisfaction on occasion of being told that my complaint had resulted in the termination of the spammer's account with their ISP.
That's not very practical anymore. The volume of spam has gotten too high to justify spending that much time on it. I use Outlook to pick up my email, and the junk mail filters are pretty aggressive. They also have problems. A great deal of junk mail still slips through the filters, and occasionally the filters block legitimate mail. So, once a week or so, I open my junk mail box and read through at least the sender's name and subject lines to make sure that email I really want isn't being trapped, then I flush the entire junk mail folder without ever reading anything in it.
Outlook's junk mail filters aren't that great, so for a while I tried using Inboxer. I was never very satisfied with Inboxer, so eventually I replaced it with McAfee SpamKiller. The nice thing about SpamKiller is that it will download new spam filters periodically. The sad thing is, it's still not much better than the Outlook junk mail filters. So I just deal with spam the hard way. I've become resigned to it, and it's still a waste of time.
When I started blogging, I began by using LiveJournal. I wasn't satisfied with their service, and eventually abandoned my LiveJournal, and after a while, I deleted it and the account. Then, for a while I started an MSN spaces blog. It didn't take me too long to decide that that wasn't exactly my cup of tea either. All during this time, I had been working on a motorsports website. Well, I "say" working, but it was more the occasional bout of coding, followed by boredom.
Most of my free time on weekends was consumed with racing. Just about every Saturday we'd take the car out to the track and spend what time we could tuning it, and waiting for a chance to race. Every now and then, it would inspire me to go work on the web site, but it consumed most of my free time, so work on the web site languished.
This summer, I dissolved my racing partnership, and tore down the old motorsports web site. I started this web site, and began by building a few features here and there, and working on the weather forecast section of the website. Then, at work, I started doing a bit of research into RSS and as a practical exercise I decided to try applying an XSL transform to a few of my favorite news feeds and hooked them into the site. It worked out OK, so I took that effort and applied it to a few things at work.
Anyway, one thing led to another, and I started using RSS feeds to simulate a blog. I went back to MSN spaces and created a new Blog, and for a while I rendered the RSS feed from that weblog on this site. Playing around with it some, I came up with a layout that I liked. While that was going on, I discovered Windows Live Writer, which is the tool I am currently using to post to this blog.
I also encountered my first Trackback spam. Mostly it was for pharmaceutical sites. It was still annoying, so I turned trackbacks off on my MSN spaces blog.
A little bit of research into Windows Live Writer, and I learned about blogging APIs. I studied them a bit, and downloaded the source code to an XMLRPC package for Microsoft's .NET framework. Within a week, I had a functional set of APIs that I could use to post, edit, and delete blog entries, and I had it working with Windows Live Writer. At that point, I transferred the last several posts from my MSN spaces blog to this one, and abandoned the old MSN spaces blog once and for all.
One feature that I've seen on just about every blog I've ever read is a comments module. So far, my favorite is the threaded comments provided by LiveJournal. Anyway, I wanted comments for my blog, so I started looking at how they were implemented on a few of the other blogs I read. My original intent was to implement my own comments UI, but I decided that I would use the Haloscan comment service to begin with. When I signed up for an account, I learned that Haloscan also supported trackbacks, so I signed up for both features.
With Haloscan I never had any problems with comment or trackback spam. That was nice. I was also quite pleased with the service's regular emails that notified me when new comments or trackbacks were received. I still wanted to implement the features for my blog myself though, so I studied the Trackback API and I implemented a trackback module for the site. Once it was implemented, I dropped the use of Haloscan trackbacks, although I kept their comment support.
Almost from the first day that I implemented trackbacks, I started receiving them. My first trackback was from Butch over at 123Beta. Every now and then, someone else would ping the trackback module on my site, and I would notice it when I reviewed my latest posts. What I didn't realize was that the spammers had discovered my site. Between the first and second legitimate trackbacks there were somewhere between 30 and 40 trackbacks from spammers. I wasn't even aware of it at first, because they hit a page way back in the older posts.
What drew my attention to the trackback spam was another feature I implemented early in the development of the site. From time to time, exceptions to the normal processing flow happen. Sometimes it's because I made a programming error, and sometimes it's because of some other condition, like an unexpected timeout or a crash in SQL server or some other problem that I didn't anticipate. Anyway, when these things happen, the site code generates a stack trace, enumerates all of the server variables, session variables and the like, and packages it together into an email message and sends it to me.
Usually there are three or four of these messages in my inbox per week. Most of the time they come when googlebot or slurp! comes and crawls my site and tries to reference a page from the old motorsports site that no longer exists. On this particular day though, there were about 50 of these messages in my inbox, and they were all generated in the space of five minutes. On closer examination, I found that all of them were caused by the misuse of the trackback API by a particularly inept spammer.
Now I'm not egotistical enough to think that I'm the first person to encounter this problem. In fact, I spent some time reading about it and heard about several anti-spam filters for different blogging systems. The problem is, I wrote the software for this blog from scratch, and I didn't use any of the popular blogging engines, so none of the anti-spam filters I've heard about will work with my site.
So I implemented a brute-force solution. First, I added new tables to the database to hold banned IP addresses. Then I modified the trackback code to reject trackbacks from any IP address in the tables. Then I added the IP address the ping was sent from for every trackback I received to the banned IP address table.
That didn't stop the spam, but it did slow it down. So I added a track-back trap to the original post that was getting most of the spam. Within a couple of days the trap had banned an additional couple of hundred IP addresses. Then the spammers discovered another post, so I added code to the trackback module to retrieve the page referenced in the trackback ping and analyze it for a reference to the permalink of the post being pinged, and I took out the auto-banning trap.
That seems to have been effective. I haven't had a successful trackback ping from a spammer since then, but people that have linked to my posts before pinging them have been able to trackback to them successfully. Shortly after that, I finally implemented my own comments module. Now the only thing I use Haloscan for is to send a trackback ping to other sites. I'm still working on the auto-discovery routines for trackbacks. When I'm finished, if a post of mine includes a reference to a post on someone's blog that includes the appropriate embedded RDF for auto-discovery, my posting module will automatically send trackback pings as soon as the post is published. That's a couple of weeks away though.
Anyhow, once I implemented my own comments module, I started running into comment spam. Today I modified the comment module so that it will send me an email with every comment received. The email contains embedded links into the site that allow me to delete comments from spammers, and to ban the IP addresses of egregious spammers if I choose to do so.
I've already had the opportunity to use it, and it works pretty much the way I expected it to. And so, that's my first "good thing" for today (well, yesterday now). I love it when code works the way you want it to right away.
This morning I received an invitation to join the Wednesday's Heroes Blogroll group. This was a no-brainer for me. I support the United States military and the young men and women who volunteer to serve whole-heartedly.
I am proud to be an American, proud to come from a military family, and proud to support the troops. Due to some rather poor choices that I made as a young man I never took advantage of the opportunity to serve in the military. I had several opportunities and I wasted them. Now I'm too old to join, and I regret it.
I'm grateful to be given an opportunity to honor the men and women who serve in the military to defend our country and to serve her interests. If by sharing their story with my readers I can help to honor them, I am happy to do so. Without them, the freedoms I enjoy would probably not be available to me today.
To all of the men and women who serve or have served in the United States armed forces I have one thing to say. Thank You. Thank You Very, Very Much.
Finally, this will probably be the last of the "Three Good Things" posts for a while. It has been an interesting (to me at least), if short, exercise. I've found that at times, it's hard to think of even three good things that have happened during the day, but that the effort is worthwhile.
I've learned that often, my life simply falls into a routine of work, reading, and a bit of quiet time at home. All three of those things are, in my opinion, good things. I enjoy my work, I enjoy reading and thinking about the news of the day, and simply reading for the fun of it. I also really enjoy quite times at home.
Writing about that all of the time though would probably become a chore. After all, how many times can I write about a good meal, or how wonderful it is to have the love of my family and say it to my readers in a way that's interesting? It's not any less true because I write about it time and again, nor is it any less true if I don't write about it.
The thing is, a lot of the good things that happen in my life are the very "ordinary" things that happen to me every day. The very same sort of things that I'm sure happen to just about everyone out there. Writing about them has served as a reminder that they are, in fact, good things in my life. I don't need to write about them to be happy though. I'm already happy.
Sure, there are a lot of things that irritate me. But those things don't really make me unhappy. What makes me unhappy is when I fail to live up to my own expectations, or when I let someone down because I wasn't thinking things through to carefully. What can I say though, I'm a sinner, and I know it. We all are, and denying it, or even trying to compensate for it through "good works", charity, or anything else we do to assuage our guilt won't change it.
That's not to say that trying to do the right thing isn't worthwhile, because it is. That doesn't say that it's OK to just give up and forget about doing what's right either. There's a reason we feel guilt when we fail to do what's right after all.
A lot of people get depressed around the holiday season. It's called seasonal affective disorder (SAD) or something like that. Some psychologists say that it has a lot do do with the reduced lighting available with shorter, cloudier, grayer days. I think part of it is because no matter what we do to try to salve our guilt at our own failings, it's never enough.
As I said, I'm a sinner. I'm not very religiously observant. On a reading of Dante's Inferno, I find that there isn't a single one of the nine circles of Hell that doesn't hold a place for me.
That's one of the things about Christmas though that I really like. The ACLU may get upset that schools allow the singing of Christmas carols or that Santa Claus is included in public celebrations. They may try to remove nativities from public property. The more they fight the true meaning of Christmas, the more public the fight, the stronger the reminder to us all that even though its a secular holiday, Christmas is really about Christ.
He came to be one of us. He was born into lowly circumstances. He suffered as we do. He was tempted as we are. He overcame it all, and in the end He sacrificed Himself to atone for our sins. This is what Christmas is all about. Accepting His sacrifice and living in gratitude are the true keys to happiness. At least they are for me.
Trackback URI for this post: http://perrinelson.com/track.aspx?postid=260
Permalink URI for this post: http://perrinelson.com/2006/12/7/260.aspx
Subscribe to this entry's
comment feed. (Atom)