Wednesday, June 22, 2005

The One in 1 Million False Positive Rate

I’m a little late with this week but hey, it’s summer: golf, swimming, baseball and an occasional blog. So here’s the “occasional” blog.

From a Ferris Research item, “Spam Control--Realistic Performance Today” came the following…

It may be true that a given vendor can achieve a one in 1 million false positive rate, but if so, it does so at the expense of letting more spam through. The higher the proportion of spam that you catch, the higher the false positive rate”.

True! True! True! And in case I didn’t make my point… TRUE!

This enforces what I cited in an earlier blog titled “The 1/3 Rule of Life (and spam)”. How well an administrator tunes their corporate antispam operations (server-based software, appliance or off-site service) determines how well a spam filter works.

CMS studies have shown that training Bayesian filters on actual messages greatly reduces false positive rates. Last time I looked Praetor’s false positive rate inhouse at CMS was around 0.027%.

Since we know that everything requires administration, how easily a Bayesian filter is trained is a high priority question.

  • Is training done in-house or must spam messages be forwarded to an outside service provider for processing?
  • Is Bayesian training done immediately, as you require it, or are your spam samples put into a queue with other companies and organizations?
Let me re-enforce these points. Believing that you get “one in 1 million false positive” indicates…

  • You are dreaming
  • You are flooded in spam

AAS

0 Comments:

Post a Comment

<< Home