SMTP quota & sendmail

SMTP quota & sendmail

Posted by: Jamin
Posted on: 2006-05-10 09:54:00

In reading these forums and the DH wiki, I understand that there's a 100msg/hr limit on sending mail. Does this apply only to actual SMTP usage (Thunderbird, Outlook, whatever), or does it also apply to sendmail/postfix?

We run a fundraising company where our member organizations can send an email to everyone in their "fan club." These emails are sent with PHP's mail() function, which I assume is going through sendmail/postfix. Currently we're quite small, so I don't anticipate ever actually hitting that limit any time soon, but as we open organizations in new markets it may become a problem. I'm wondering if I need to set up some sort of email queue and have a cron job to send out X number of emails from the queue per hour, in order to avoid the limit.

Thanks in advance. :)

---Jamin

Re: SMTP quota & sendmail

Posted by: rlparker
Posted on: 2006-05-10 17:25:00

That is a great question, and I have been wondering/worried about that myself. As I have written in several other posts, I see this issue as being *huge* for interactive sites that rely upon email notifications for sign-up activation notices, order confirmations, notifications of new posts/articles, etc.

This could be a deal breaker for manyh of us if reasonablyeasy to implement queing systems, or processes of some other types, are not forthcoming. It would suck to host at DH till you got "hot" then have to leave at the peak of your "surge" so you could communicate with your users.

I'd *love* to hear Dallas's take on this concern!
--rlparker


Re: SMTP quota & sendmail

Posted by: dallas
Posted on: 2006-05-10 17:52:00

There are separate limits on authenticated smtp sending (via Outlook, Thunderbird, etc) and web server machine email sending (using php's mail function, direct pipe to sendmail, Pine, Mutt, etc). There are limits on both and the limits are set system-wide and can be adjusted on a per-user basis. The limits have become a necessary evil to help us fight spam emanating from our servers. Currently website users are allowed 200 emails per hour (that's per user, not per website), but that is subject to change (as are any of our policies).

The limits on the web server email sending were put into place first to fight spam coming from exploited web accounts. The attackers hijack email scripts or get access to the website through an exploit in software on the website. This still happens many times a day but the damage is now significantly reduced.

The limits on the authenticated smtp were put in place due to malicious people signing up for accounts and sending out spam directly. They all use stolen credit cards and sign up with fake information and generally use AOL accounts (though we have seen others). We can't just block all AOL users from sending email through our servers, unfortunately, though it has definitely been considered.

One key piece of information here is the limits are set per user so they can be adjusted if there is sufficient need for that. We're not trying to prevent you from running your website.

These limits are in place to safeguard our network as much as possible from being blocked by other email providers and also to protect your accounts from being hijacked and used to send spam. It's much better for everyone involved for a hijacked account to only send out a few hundred spams instead of tens or hundreds of thousands.

- Dallas
- DreamHost Head Honcho/Founder

Re: SMTP quota & sendmail

Posted by: Scope
Posted on: 2006-05-10 18:08:00

ive referred a few folks recently and setup a blog where visitors can opt-in / opt-out of email notifications when a new post is made to a blog (tied in with wordpress)... subscribers are already close to a hundred after ony few weeks.

when was this change made? nothing mentioned on dreamhoststatus.com

Deinitely will lose a few folks this way...and i feel foolish for having refered them.


Edited by Scope on 05/10/06 06:19 PM (server time).

Re: SMTP quota & sendmail

Posted by: rlparker
Posted on: 2006-05-10 22:57:00

Thanks, Dallas, for taking the time to explain the limits more thoroughly then I have seen so far, and for sharing the reasoning and goals behind them.

In reply to:

One key piece of information here is the limits are set per user so they can be adjusted if there is sufficient need for that. We're not trying to prevent you from running your website.


I, for one, am encouraged most by that statement. I'm choosing to read that to mean DH will work with us if legitimate mail needed to operate a site should run up against those general limits (assuming they have not already maxed out CPU usage before that happens - tongue in cheek bit there). That seems fair enough to me, especially if there is a mechanism to get it adjusted "quickly" when Oprah of Slashdot sends half the internet to one of my sites (Yea! Oh... sh*T! OH! S*(*&^%T!!!)

Some of us (lazy? and I *am* referring to myself here) should probably reconsider running numerous websites under our "master" (or any single) user. It was *so* convenient to do that, but the "by user" limits means it would make more sense to run *each site* as it's own user, in order to avoid running up against the email (and, I suppose, CPU!) limits.

Your sense of "fair play" is to be commended :

In reply to:

We can't just block all AOL users from sending email through our servers, unfortunately, though it has definitely been considered.


.

Now that is a great example of "taking the high road", or "being the bigger person" if I ever saw one, as AOL seems to have had no qualms about blocking email from entire DH servers frequently and repeatedly. I hope that "what goes around comes around" for DH with that "enlightened" attitude. You guys reallly do "rock", and deserve a little (lot!) more respect from some of these other providers for all that you do to keep spam from hitting them from DH.

In reply to:

These limits are in place to safeguard our network as much as possible from being blocked by other email providers and also to protect your accounts from being hijacked and used to send spam.


Reasonable, and noble goals to be sure, and probably necessary to a greater degree now than a few years ago at DH, given the hugh influx of new users and their scripts. It is also worth mentioning that we are given *lots* of freedom to run scripts of many kinds and of greatly differing quality. I *think* I would rather have these kinds of "safeguards" in place, and be able to enjoy greater freedom to run what I want, then to be restricted to a "select handful of approved scripts", which I think would be the only reasonable alternative to such safeguards.

Maybe, if the support burden wasn't too great, as an alternative to the email limits, DH could "approve" a site's software and , in conjunction with some other "vetting" (such as length of time with DH, track record, etc.), selectively raise/remove limits *before* they get maxed out.

In reply to:

It's much better for everyone involved for a hijacked account to only send out a few hundred spams instead of tens or hundreds of thousands.


I absolutely agree with that statement! No one can authoritatively state that their site/script "cannot be cracked". No slam intended at any of you "wizards" out there, but everyday brings a new threat. I ran a "safe" (and very popular around the net) php formmail script for almost a year without incident, as did many others, but the day came when the crackers found an way to exploit it, and I had to dump it. Better that DH limits the damage done in such a circumstance than have us all suffer the consequences when the thing breaks down.

Dallas, thanks again for taking the time to post. After reading your post, I'm not nearly as discouraged by the email limits as I was. I hate to add another bunch of users to DH, but it seems the way to go...Now, for a whole lot of re-configuring to make each of the sites I manage run as its own user...how will I ever keep all those users/passwords
straight ;-)?

--rlparker

Re: SMTP quota & sendmail

Posted by: rlparker
Posted on: 2006-05-10 23:08:00

It sounds like you might already be approaching "trouble territory" if I understand Dallas's post correctly:

Lets' say 80 current users, with half of them signed up for email notification of new posts, and 5 posts in an hour (40 * 5 = 200).

Even with no other sites, or other email, you would be "at limit" right there! Or , another (maybe more realistic) way of looking at it: you grow to have 210 users, and you post a new entry -> Over Limit!.

Does WordPress have an email queing system that can "spool" the outgoing mails and dole them out "N" per hour, or some other email control mechanism?

--rlparker

Re: SMTP quota & sendmail

Posted by: tez
Posted on: 2006-06-12 19:28:00

Can't dreamhost just queue these for us? It is going to be a real pain in the A to code our own mail queuing systems for every mailing script we have.
And could dreamhost whack on a Control Panel tool to check our outgoing queues and allow cancel of outgoing email ;) that would be sweet. That would be just another spot account hijacking - unless there is something bad about seeing a list of outgoing mail?

Edited by tez on 06/12/06 07:30 PM (server time).

Re: SMTP quota & sendmail

Posted by: jeffweese
Posted on: 2006-06-21 06:24:00

This is the answer I got from support on this issue:

Hello Jeff,

On Mon, 19 Jun 2006, you wrote:

>
> I was wondering if you can adjust the SMTP quota on a case by case
> basis? I don't often send out a large number of emails but
> occassionally I have the need to email all the members of my local
> Realtor association which is 370 members.
>
> I appreciate your anti-spam efforts and support them fully. But it is
> a pain to have to email in groups of 100 or less. It will take me
> all day before I finally get my email to everyone.

If you plan to use this account for bulk email purposes, we will first
need to ascertain that your usage is consistent with our anti-spam
policy, which you can review here:

http://www.dreamhost.com/spam.html

The most important aspects of this policy, which we would like you to
address, are:

1. Whether or not you have used an opt-in confirmation process (not to be
confused with regular 'opt-in') for all of your your list's subscribers.
This is a process wherein each person who signs up for your list is sent
an email after subscribing with a tagged link in it that they must click
on before being added to the list. Those who do not click on the link are
not added, and receive no further bulk email.

2. Whether you log each confirmation with the date/time and IP address
associated with that confirmation.

3. Where we may go to independently review your opt-in confirmation
logging data.

Also, we would like to know where we may sign up for your mailing list in
order to test out its opt-in confirmation functionality.

If your bulk email usage is not in compliance with these policies, we
must ask that you cease any bulk email activity immediately until you
have become compliant. The easiest way to do that is to use our
Announcement List feature, which you can access here:

https://panel.dreamhost.com/index.cgi?tree=mail.list

The Announcement List feature handles the opt-in confirmation and logging
aspects of the policy for you, so that you don't have to.


Thanks!
Mike S.

Re: SMTP quota & sendmail

Posted by: ugotme
Posted on: 2006-06-26 21:29:00

I'm running a web site for a political campaign and using ListMessenger (www.listmessenger.com) to communicate with constituents in our county. Tonight while sending an email message to about 1400 voters about a candidate forum meeting happening this Wednesday, I received about 200+ emails back with the error message SENDER_QUOTA_REJECTION.

How can I adjust the quotas for this user? Alternatively, should I try to space out the delivery to less than so many messages per hour when using the PHPmail function? If so, what should that number be, and can I have it increased from the default for this particular user?

We are complying with the requirement to have an opt-out link in our email messages, and we provide the campaign committee's name, address and telephone number in each email. All recipients are registered voters.

Please help!!!

Thanks,

AJ

Re: SMTP quota & sendmail

Posted by: sdayman
Posted on: 2006-06-27 06:38:00

Contact support, and be prepared to provide the information outlined in the post immediately above yours. Someone quoted Mike S. on what Dreamhost wants to see.

-Scott

Re: SMTP quota & sendmail

Posted by: halavais
Posted on: 2008-08-26 22:02:00

This is really just a ping to reiterate the original question. I have a couple of systems that need to generate mass emails. An academic conference (running OCS) sent out acceptances that didn't make it out, resulting in a lot of confused people, and a messed up conference. Don't want to see that happen again. (In each of our cases, the customers are paying for registration, and so are unlikely to be spamming email addresses.)

Unfortunately, creating new systems to do email verification and then report it to Dreamhost represents a significant expense of time and potentially money for us, and so the use of a spool makes more sense. We don't care if our 600 emails go out instantly, or over six hours. So, I'll re-ask: does anyone know of a sendmail spooling script already in existence?

I'm afraid it's a bit at the edge of my ability. It doesn't seem hard to set up a script that accepts emails and queues them if too many have gone out, but I'm not entirely sure how to get php mail to redirect to that script (to avoid having to root out all the calls in the scripts that use it).

Tags: quotasmtp