Friday, April 18, 2008

Cool Technology of the Week

I've written about the challenges of Spam filtering - false positives and false negatives.

Recently we've experienced a doubling of the volume of incoming Spam. It's essentially a "Spam Denial of Service attack" that is overwhelming our Spam filters. The filters have a failsafe behavior that automatically lets Spam through if the servers get overwhelmed. Leaks of Spam and the increasing challenge of providing reliable, secure, 99% spam free email has caused us to revisit our email configuration and spam filtering products.

Our Spam filtering company, Symantec, provided onsite engineers to examine our configuration and hardware design. They suggested reconfiguration, enhancement of our CPU capacity and an upgrade to the latest software version that "learns" about common email patterns within the organization and whitelists selected traffic, relieving the burden on the spam filtering servers. Symantec also suggested replacing our software-based product with their 8300 series appliance. The appliance is better equipped to process large volumes of mail.

As a class of technologies, Spam filters include pattern recognition, Bayesian probabilistic decisionmaking, and neural network techniques among others. The best comparison of Spam flters, I've found is a recent Infoworld article.

The article illustrates the difficulty in improving our situation. The Symantec product comes out best in class, but only stops 96.4% of Spam. There were products that did better, but most had offsetting problems with false positives. Only Sendio and Proofpoint had better Spam blocking rates and no "critical" false positives. They both had much higher "bulk email" false positives than Symantec which accounted for the "best in class" rating for Symantec. The Infoworld evaluation was based on the Symantec appliance. The appliance has the same anti-spam engine as their software, but can perform additional functions e.g. better reporting, smtp-based throttling based on locally observed reputation, and others. We are testing the appliance now.

Our challenge is that as a healthcare provider, we cannot have false positives. A critical patient email, lab notification, or followup from a medical colleague must be delivered. We will accept a bit more Spam in order to have few false positives.

Thus, for now, we've concluded that an appropriately configured, hardware optimized Symantec configuration is our best bet. The war against Spam is a continuous battle, but for now, 96.4% filtering with very few false positives, wins the race. Hence, Symantec Anti-Spam (formerly known as Brightmail) and their 8300 series appliance is my candidate for the Cool Technology of the Week. Spam is an elusive target, so we'll continue to watch the efficacy of all the available products.


Jonathan Merrill said...

Have to disagree on this cool technology. As a previous user of Symantec products, their engineers were virtually non-responsive to our virus breakout at two of our hospitals. After multiple phone calls to Symantec pleading for help and being told they would call back, we finally downloaded a trial of ESET NOD32 and began deploying it on our server backend.

NOD32 immediately found the offending viruses and cleaned us up. A couple days after this incident, Symantec released DAT files to "detect" (not clean) the trojans that comprimsed the machines.

If this was a isolated incident, I would have given Symantec the benefit of bad timing. But in my 17-year IT career, this exact situation happened at three other locations with the same result.

I've subsequently lost confidence in Symantec and their products. We've been using ESET NOD32 on servers and desktops with resounding success. For email, we've been very pleased with Sun-Belt Software's Spam Ninja, a truly second generation spam detection and cleaning tool.

I felt compelled to write this comment as I am baffled at the continued success of Symantec. I feel their products stand the test of time not from technological superiority, but from sustained marketing and brand familarity.


JGF said...

We're slowly moving towards authenticating the sending service, and incorporating the reputation of the sending service into Bayesian rules.

I recommend asking your Symantec engineers about they're use of these standards in their decision making. If they're not using them, then I'd look more seriously at other vendors.

If they are using them I'd be interested in learning that as well.

Anonymous said...

Hello. This post is likeable, and your blog is very interesting, congratulations :-). I will add in my blogroll =). If possible gives a last there on my blog, it is about the TV de Plasma, I hope you enjoy. The address is A hug.

John Halamka said...

Regarding anti-virus products, we use Mcafee throughout the enterprise. The product we use for spam filtering, Brightmail, was acquired by Symantec and is a new division for them. The new version of Symantec does incorporate Bayesian techniques to whitelist commonly observed mail routing patterns

Unknown said...

Hey John, this is Angelos Kottas from the Symantec Mail Security team. This probably goes without saying, but I like your post! I'm glad we’re helping make your life easier, and I hope we continue to meet your needs.

I’ve been following the comments here and wanted to jump in real quickly to address a couple of John Gordon’s questions and hopefully do so without coming across as too salesy. The first point is that we don't take a single approach to combating spam, and we've definitely continued to innovate since buying Brightmail. We combine 20+ different spam filtering technologies in our solution. This includes our global intelligence network (blocking connections based on Zombie Lists and other known spammers), the adaptive local reputation approach you mention (which determines potential spammers based on actual email traffic observed at a customer site), URL filtering for malicious, compromised, or inappropriate web destinations, a variety of heuristics including content analysis, structural analysis, and image-based analysis, and a comprehensive list of signatures that are updated every 5-10 minutes.

Our spam “rules” and signatures are generated by Symantec's global back-end resources, which include both automated filter creation as well as a team of spam analysts who are devoted solely to this. Obviously we feel like our approach is best, but its great to see customers and 3rd parties like InfoWorld buying into the results.