Just wanted to apologize for all the weirdness last week. I wrote this in my announcement about the problem, but a lot of these issues are difficult to ferret out when testing. Things can work well when filtering a few messages per minute and behave somewhat differently when filtering a few per second.
Dealing with a very broad-based, network-level email filter is always a delicate thing. That being said, the problems last week were really dumb and were my fault.
In reply to:
I think the mail was fully registered at quaratine (and hence would be also in the backup malbox) just that mail went missing only upon the Move to Mailbox. Which frankly doesn't surprise, given JF is carrying out this move by the shockingly manky method of forwarding the message to INBOX, instead of using e.g. a fail-safe IMAP move.
Well, that's not really how it works. The "Junk Folder" itself is not a real folder, it's not stored on your mail machine, and doesn't speak IMAP. It would be a billion times easier for me on the Webmail side of things if it were. However, storing the Junk along with all your other mail on our mail servers defeats a lot of the purpose of the filter: to keep this crap off our servers in the first place.
The mail is (eventually) stored as a file on a network file server (which does have backups, just like user storage). The headers are scanned and info is put in a database: that's what you're viewing when you click on the "Junk Folder". It's just info from a database. For obvious reasons, the body of the message is not stored in the db.
The problems from last week stemmed from a bug in the process that takes mail the filter has determined is spam and scans the headers for the database. It was a stupid bug that only exhibited under a very heavy load. It's fixed now.
Again, apologies all around. I'm always doing everything I can to make sure this stuff both keeps spam out of your Inbox and doesn't muck up your "real" mail.
nate.