when released, will catchall's work?

I've been using catchalls for quite some time and am dependent upon them (though I have begun migrating away from them.) When the new filter is released, will it still have the limitation of not working with catchalls?

Maybe Nate can comment further, but I'll try to respond here since I've talked with him about this a little.

My understanding is that we will require people not to use catchalls to use the new spam filtering system (even after the beta). Even /without/ spam filtering, we've had huge problems come up from people with catchalls (HUGE volumes of bounce-storms from forged spam runs, dictionary attacks, high volumes of spam, etc.). This could cause delays in the whole system for everyone.

With ~ 80k domains, and probably (off the top of my head) about 120k actual "mailboxes", and growing, and with (these days) at least 50-70% of incoming mail being spam, we're talking about a non-trivial amount of hardware / processing power to filter all those messages. This means we want to do anything we can do to reduce the likelihood of problems which will cause email slowdowns / instability for large numbers of (other) users.

Catchalls are not always a Bad Thing, but in this case, I think we may need to keep the requirement that people don't use them along with spam filtering (though we might be responsive to requests for features to make it easier / quicker to add tagged addresses). Nate wrote a cool little script for himself to go through his email and identify the email addresses at his domain (that used to have a catchall) and create aliases for them. This will admittedly be harder now that we don't have an "expert mode" for adding addresses / aliases. That's something we can work around, though (and if you have a whole lot, we might be able to bulk-add a bunch for you directly in the database).

Side note - if you're going to setup a catchall, you may want to set it up at a random subdomain rather than at your main domain. Spammers are at least a little less likely to forge addresses at some.weird.subdomain.example.com than at your domain itself.

SPF, of course, /should/ help a little with dealing with spoofed email and bounce-storms... but I'm not quite holding my breath yet.

I'm not sure exactly how we're going to deal with spam filtering for addresses which forward off our system completely either - that's going to be a little tricky because obviously it's a good idea to spam / virus filter the mail before forwarding it on, but it's hard to quarantine (and unquarantine) messages when there's no actual mailbox or login to our system.

> I'm not sure exactly how we're going to deal with spam filtering for addresses which forward off our system completely either - that's going to be a little tricky because obviously it's a good idea to spam / virus filter the mail before forwarding it on,


Actually it is not necessarily a good idea to spam/virus filter email before forwarding because some of us (e.g. me!) deflect our email to other (non-DH) systems where we are testing and writing about spam and virus filtering tools. So please make it an option to relay mail to another system without snagging any of it or editing any headers (except of course adding a Received and Delivered-To header).

Thank you,
Nancy
http://email.deflexion.com

Nancy
Infinite Ink ~ http://www.ii.com
Deflexion & Reflexion ~ http://deflexion.com

In reply to:

---

So please make it an option to relay mail to another system without snagging any of it or editing any headers

---

A good idea in theory, but with the current situation (of providers filtering mail their customers indicate as spam), this probably isn't workable.

Now of course any filtering options we have will be configurable to a certain extent, but if there's anywhere we want to strongly encourage people to have some filtering, it's when forwarding to outside addresses.

Yeah, what Will said. There were also minor technical annoyances for me if I was going to make catchalls work with the Junk Mail preference database. That didn't make it a very hard decision.

During the first beta test, there was a guy who had a catch-all. The filter was quarantining 4,000 spams a week sent to the catchall mailbox. That is insane. The processing power it takes to handle even a few hundred mailboxes like that (out of the 156,288 mailboxes we host) would be extreme, to say the least.

And I thought my piddly few 150 spams a day were bad.

Catchalls are sometimes convenient, but it's a just a laziness thing for almost everybody. Very few people have a real application that requires catchalls.



nate.

So when this system is released, will Razor still be available?

I started using DH in 1997. I didn't even know what a catchall was until I saw DH configured with them on by default. I started using them as an antispam technique. Basically, if I went to a site and had to provide a valid email address, I would give (for example) CNN the address cnn.com@example.com. I've done this possibly for hundreds of sites.

With the recent domain-spamming viruses, I've seen a huge increase in spam (obviously.) I have started using, for example, cnn.com+spam@example.com (and using a subdomain is also a good idea.) In this case, spam is a real mailbox.

I'm stuck with catchalls for a while.

In reply to:

---

So when this system is released, will Razor still be available?

---

Probably for a while, at least, and we'll likely keep the programs installed even if the panel option goes away.

In reply to:

---

Probably for a while, at least, and we'll likely keep the programs installed even if the panel option goes away.

---

Will it be updated and maintained?

In what sense?

When updates are released, will DH be installing them?

For some reason I feel like I'm about to be left out in the cold.

Not sure about the turning off the virii detection but you can set your email spam level to 999 so it detects basically nothing..

Well just as much as we do currently - we would update it when the Debian version changed (which would generally be when there's a new major release).

I imagine that if we were to remove support for it, we would also remove the mechanism for people to move emails to a folder to have it submitted as blocked-non-spam or unblocked-spam.

I honestly don't think too many people would be crying about the demise of Razor, though.

For me, it's better than nothing. I've been pretty good at keeping my main addresses spam-free. I'm just worried DH will drop razor and leave me without any protection.

"Nate wrote a cool little script for himself to go through his email and identify the email addresses at his domain (that used to have a catchall) and create aliases for them. "


Anyway to get a copy of this script?

-oZ

> "Nate wrote a cool little script for himself to go through his
> email and identify the email addresses at his domain (that
> used to have a catchall) and create aliases for them. "
>
> Anyway to get a copy of this script?

I hate to be a 'me-too-er', but I'd like a copy of this script as well. I also use a different email address at each site I visit, so trying to locate all of the addresses (such as papajohns@firepunk.com) would be messy.

thanks! brandon.

Is the load problem caused by the quarantine process, or is it a case of not wanting to scan addresses that receive a larger amount of incoming mail?

If it's the former, one solution might be to disallow quarantining in domains with catchalls, but still allow subject tagging.

Quarantining is cheap. Just slurping text into a database. Scanning is really expensive. It's because of the amount of incoming mail.



nate.

"Very few people have a real application that requires catchalls."

What I liked about catchalls: I could "invent" an email address on the fly -- at a store, at a conference, online while purchasing a product, at a bar, whatever -- and wouldn't have to plan ahead. ("Oh hi Julie, nice to meet you. My email address? "datejulie@mydomain.com.") That was one of the biggest advantages I enjoyed from having my own domain name in the first place.

I'd get the mail at the new address, and if I ever started getting spam at that newly-made-up-address weeks later, or wanted to avoid Julie, I could easily blackhole it.

Without a catchall now however, there is forethought needed. I have to create the accounts ahead of time and can no longer make them up on the fly. If I DO invent a new one spontaneously for a particular purpose, I need to rush to a computer to create an mailbox so the mail doesn't bounce back.

Am I missing something or is there some other way to approximate this same convenience?

you can use a +

I mentioned this in my above post from 09/03/04 12:19 PM

now you would use datejulie+spam@example.com or bikerbar+spam@example.com - with spam@example.com being a real email address.

It would usually be the other way around (user+extension). In this case, you can create a .forward+foo file to put mail to user+foo@example.com in a different folder, forward it somewhere, or send it to the bit bucket (i.e., junk it). However, this won't work with our current setup.

When we switch to Postfix v2, which supports user+foo style addressing for virtual domains, we may enable this feature.

It seems like a smart spammer would know to strip the +extensions in order to cover their tracks. (similar to how they currently remove decorations like "nospam")

One sneaky use of catchalls I've been playing with is encoding a datestamp and IP in an email address on my web site, to see where the spammers are coming from. :) I might also use it to expire addresses after a month. (That's on a throwaway subdomain, though.)

In reply to:

---

It seems like a smart spammer

---

See Rule #3...
http://bruce.pennypacker.org/spamrules.html (spammers are stupid).

Strictly speaking, that's not /always/ true... but I think it's fair to say that most spammers are not the sharpest crayon in the box. In most case, spammers don't bother to try to clean up harvested addresses at all. That would involve work. Plus, the type of people who are tech savvy enough to use plus style addressing are probably not most spammers' target audience anyway. I have seen some funny results from attempts to remove "nospam" type munging.

Actually, one of the widely posted addresses I get the least spam to has &- as its LHS (left hand side - part before the "@").

now I'm confused. Why is it that if I use foo+user@example.com, the email goes to the inbox of user?

to the "what if spammers are smart question" - that's the beauty of mine. If they are smart and search for "spam" or "nospam," then my address of cnn.com+spam@example.com would be invalid :o)

foo+user will not go to the inbox of user on most systems I'm aware of.

user+foo will on certain systems if that functionality is enabled.

If you're showing this behavior on our system, most likely you have a catchall setup which goes to "user", so it's just working because there's a catchall.

NOOOOO!!! Looks like you're right. I thought I was migrating away from catchalls. *sigh*

So what's a good way to create "disposable" email addresses? Looks like the suggestion of using a subdomain with a catchall might be the best... hmmmm.

The best way to make disposable addresses is to use a third-party service such as spamgourmet and keep spammers as far away from your "real" domains as possible. :)

gmail.

;)

Want an invite code?

Given how well spamassassin is working, honestly I feel like you guys are trying hard to find a solution for a problem that doesn't exist.

How can gmail do it? Don't I need a different account for each address?

As far as spamassassin, it's pretty much been said that it will never work with anyone who uses catchalls. I've been trying to get away from them, but I still have to support them.

You know I've actually stopped using throwaway email addresses. Not only was my catchall receiving a lot of spam, but I had never even deleted an address that started getting spam.

I use SpamProbe (a statistical filter), not SpamAssasin, and my filter is way, way better than SA, but still, I think SA would do a good enough job if I had to use it. If you are smart and really hate spam, the right statistical filter is literally like magic.

I know throwaways help track anybody who might be selling your address and whatnot, but honestly, I don't care. I'm too busy working on this crap.


nate.

There's a monetary advantage for me in knowing who sold my address. When I catch a company violating the WA spam law, they are liable for $500 per message.

It adds up.

I have to say, I have had very few situations where it was clear that a company had violated its own privacy policy - I've gotten mail I didn't want from companies I've signed up with (ticketmaster being a good example), but basically never had a case where I could prove a company had sold my email address to an outside party or sent me mail on behalf of someone else (unless that was clearly allowed in their privacy policy).

Domain registration contact addresses == big source of spam. Having a bunch of different email addresses for different domains proved that to me - but now I just get each spam a billion times.

I've been moving back towards having less email addresses, because ultimately, using tagged addresses just means you get more spam.

In reply to:

---

When we switch to Postfix v2, which supports user+foo style addressing for virtual domains, we may enable this feature.

---

Though, now that I think of it, there may be some problems doing this with the spam filtering setup as well. Gah.

Until recently, I had a client who dealt with a lot of network marketing people. My email address is on a product I provide for the client, which they resell to their clients. I gave them a unique address and soon started getting "prospecting" emails which was a no-no. Since I could prove where they had got the email address, I could point out they had agreed not to prospect with that info. Since I had not opted in, they had to stop.

I had another client who used his site to cater to network marketing people. I had an account on his system for development purposes. After a couple years, I started getting calls and emails in reference to my listing on the site. We realized someone had written a script to randomly generate usernames in order to find their bio pages, then pull off the email addresses and sell them as "warm leads."

Network marketing people tend to do a lot of illegal things (without knowing) in the name of "stepping out of the box" and just trying extra hard. Not to mention they tend to use the CC field for mass mailings and then everyone on their recipient list adds me as a "warm lead" and I start getting more and more junk.

Also, I once gave HotJobs an email address. When they were bought by Lycos, I'll just say "something strange" happened and it looks like that email address was somehow leaked. It appears they either got hacked, or got a trojan which found the email addresses.

So the spam I'm most worried about, isn't the stuff most people get. Because of this, I like disposable email addresses.

> I've been moving back towards having less email
> addresses, because ultimately, using tagged addresses
> just means you get more spam.

How do you figure? The only situation I can think of that being true is if multiple tagged addresses end up in the same database, are distributed, and some fool spammer emails you twice with the same pitch. Not sure how often that would occur, though.

Really, though, I do have to concur with some other people - if I can track down a 'legitimate' company that is spamming me in violation of their TOS, I want to yell at them or (back when I used to live in WA) nail them to the wall. :>

Have to be careful with that, though, and make absolutely sure there's a connection. We occasionally have customers who use tagged email addresses with us who email us under the assumption that we sold their information. Almost invariably, upon researching the matter we find that they used it for one or more 'whois' records as well.

- Jeff @ DreamHost
- DH Discussion Forum Admin

On the Junk Filter page, all my subdomains show up as eligible for filtering even though some have catch all addresses. It looks like the way a catch all address is specified has changed, but the Junk Filter code hasn't quite caught up yet.

I'd also like to see domains with catch all addresses supported by the new Junk Filters, for the reasons others have mentioned. But I definitely understand why DH has that restriction.

whoops, sorry! all fixed!

josh!