This Week in Spam

This week was spent comparing .Mac/Apple Mail and Thunderbird (on a Mac only).

The setup had the spam filter off at my email forwarder with email forwarded to both my Yahoo account and my .Mac account. The spam filter at Yahoo was off so all mail passed through to my Thunderbird client. The .Mac filter isn’t configurable by me so it would stop some email. Both services would receive the same email as both these services only receive email via my forwarder. I don’t give out the direct addresses for either. And much to much surprise and delight neither is attracting spam on it’s own.

During six days last week 324 emails passed through the system. This 324 number is the actual spam messages and does not include false positives marked as spam and it does not include non-spam.

.Mac/Apple Mail

While Apple Mail is not limited to .Mac I treated it as a single system for this week. To keep things consistent I only used mail from .Mac to train the junk mail filter in Apple Mail and these numbers are only for the email sent to .Mac. Of the 324 spam emails only 113 were actually delivered to my mailbox, the rest were stopped by the .Mac filters. Here’s an article (apple.com) that discusses Apple’s use of Brightmail and it’s other spam filtering techniques. The downside to having .Mac block email is that some email was blocked and I’d have no way of knowing it if it was only sent to my .Mac mailbox. These were all bulk emails and so far adding the sender to my address book has allowed future emails through. Of the 113 spam emails 108 of them were flagged as junk by Apple Mail. Five emails were missed. Apple mail had 8 false positives. Again, these 8 were all bulk emails. The percentage of missed and false positives did go down as the week progressed (as previous errors were marked and the filter was trained). But the amount of email is too small and the time frame too short to know whether this is due to improvements in the filter or just coincidence. So the bottom line for a .Mac/Apple mail combo: Spam Filtered: 319 [98.5%] (211 by .Mac and 108 by Apple Mail) Missed: 5 emails [1.5%] False Positives: 11 [3.4%] (3 blocked by .Mac and 8 by Apple Mail)

Thunderbird (Mac) Thunderbird was used for delivery of the email sent to my Yahoo account. Since I turned off the Yahoo spam filter all 324 emails came to my PC. Thunderbird flagged 291 as junk, missed 33 and had 6 false positives. For filter training all email was marked as “spam” or “not spam” each day. What’s interesting here is that the junk filter did not seem to improve as the week went on. In fact, on the last day it caught 13 but missed 15 (which was also the day that had the fewest spam messages). The day seems to be a anomaly and accounted for almost half the missed messages for the week. The bottom line for Thunderbird: Spam Filtered: 291 [89.8%] Missed: 33 emails [10.2%] False Positives: 6 [1.9%]

Spam Destinations

As I mentioned in my previous post I’ve been looking at which of my email addresses get the spam. The previous trend continues. The address I give to real people but have never personally used on a website or mailing list gets 45% of my spam. My tier 3 “I expect you to spam me” address does get 50% of my spam but on 3 out of 6 days I received more spam on my “people only” address than I did on this one.

Other Numbers

My GMail account received 21 spam messages over the week and caught all of it. There weren’t any false positives. The time period isn’t the same as the other numbers but AOL stopped 87 spam messages and let 160 through.

Conclusions and Next Week

First the disclaimer: This is only a small sample over a short period of time using a small specific sample so these conclusions won’t apply to everyone. Apple Mail has been my primary email client for awhile but I’ve always had my ISP spam filters turned on. I’m pleasantly surprised it’s doing so well especially when compared to Thunderbird. It’s time to move to the next step. Apple Mail and Thunderbird will both process email from Yahoo and we’ll see how they compare after a week of processing the same email. EMail isn’t deleted from the server so the same messages will be delivered to both clients. .Mac/Apple Mail is a good choice from the perspective of the amount of spam blocked. The fact that I can’t see what email it blocks is a problem for me. Sometimes the sender address isn’t obvious (when I register my email address) and if the email is never seen the sender can’t be added to my address book. Plus, if I don’t see it I can’t count it! .Mac alone (using only the web) wouldn’t be a good choice for me. To much spam gets through .Mac’s own filters and there’s no way to tweak it on the web which makes it tedious to manage mail via the web interface. (I do have a general bias against most email web interfaces.) I’ll be dropping .Mac from my testing, at least in the near future. Because the AOL web software crashes both Firefox and Safari on my Mac I haven’t been able to consistently manage/mark spam it receives so it shouldn’t be directly compared to the other services. So next week I’ll be working on direct comparisons between Apple Mail and Thunderbird (Mac). I’m going to also try setting up some better testing of the AOL spam capabilities.