SpamAssassin is useless

I find that DreamHost's SpamAssassin is pretty useless as it misses most spam that uses HTML or other evasion techniques. I've installed SpamBouncer and it seems to be a lot more accurate at identifying spam.

For example, one piece of pharmacy spam I got today wasn't identified by SA, as shown by these headers:

X-Spam-Status: No, hits=0.2 tagged_above=-999.0 required=5.0 tests=HTML_70_80, HTML_MESSAGE, HTML_TAG_BALANCE_BODY

That same piece of mail (I didn't turn off SA yet, so it's being processed by both) was caught by SB:

X-Spambouncer: 2.2 (Procmail 3.22 Fix) (04/16/06)
X-Sbrule: Body Domain: palisstionster.com is in SURBL (William Stearns)
X-Sbrule: Body Domain: palisstionster.com is in SURBL (Spamcop)
X-Sbrule: Body Domain: palisstionster.com is in SURBL (Wein/Dijkxhoorn)
X-Sbrule: Body Domain: palisstionster.com is in URIBL Black
X-Sbrule: Russian (iso-ir-111|koi8-r|koi8-u|windows-1251)
X-Sbscore: 24 (Spam Threshold: 20) (Block Threshold: 5)
X-Sbclass: Spam

Note that I have email addresses from several domains forwarded to a single mailbox at one of those domains. All of the domains involved have SpamAssassin enabled.

As far as spam identification technology, you are comparing two rules based spam detectors. The rule that is catching the spam in SpamBouncer is: does this message match a domain in SURBL X?

http://www.surbl.org/faq.html

Our centralized SA installation does not have network checks turned on at the moment due to the time consuming nature of some of those checks. I will bring up the possibility of turning on just the SURBL checks as that may be possible.

Of course you can install your own version of SpamAssassin with SURBLs turned on in the meantime. SpamBouncer is a collection of procmail recipes, whereas SA is a fuller application written in perl and now located at apache.org (http://spamassassin.apache.org/) so it is jumping the gun a little to declare it useless.

DreamHost's SA also seems to ignore my settings in .spamassassin/user_prefs, in which I changed the required_hits to 4 and raised the scores of many tests.

The centralized SA system we have is on a seperate machine that the mail gets routed through. You have to configure it via the webmail interface: http://wiki.dreamhost.com/index.php/Junk_Mail

That only lets me edit the whitelist/blacklist and set the overall tag level, but it doesn't let me alter the test scoring or any other SpamAssassin variables. I like to add more weight to some of the tests, since so much spam is HTML-based and the default settings always give them a very low score. There also doesn't seem to be a way to specify some of the HTML tricks like invisible text & multiple hidden DIV tags.

That is correct you can only edit the settings available on that page. The centralized Web Panel/Web Mail Junk filter is only accessible through the web interface, which doesn't allow the editing of individual rule weights.

In order to get finer grain detail you would have to use procmail to send your email through the SA install on your local mail machine.

After I RTFM & experimented a bit, I found that I can tell SpamAssassin to use my settings by adding -p ${HOME}/.spamassassin/user_prefs to the command line.

As an experiment, I've turned off SpamAssassin in the control panel and added a rule to my .procmailrc that does spamassassin -p. Will I still be able to see the junk mailbox in webmail? If not, where will I be able to see the rejected spam?

If you invoke spamassassin from the .procmail.forward file, then it should be acting as a filter. SpamAssassin will modify the message to add the scan results, and you can either use procmail to filter spam into another mailbox, or filter it in your mail client. You can't use the DH junk mailbox because that lives on another machine. Here's what I use with procmail:

:0fw: spamassassin.lock
| $HOME/personal/bin/spamassassin

# score >8 is definitely spam
:0
* ^X-Spam-Level: ********
.Spam/

# Dump other tagged spam into Suspect folder
:0
* ^X-Spam-Status: Yes
.Suspect/

I use an IMAP client to review the Spam/Suspect folder for false positives.

You should look at the messages to see exactly what version of SA is being used. Since DH doesn't support SA used this way, I don't think the installed version is kept up to date. You may want to look into installing the latest version in your home directory. (There's a good web page that explains how.)