Troubles with Dreamhost

Hi, I've been a dreamhost customer for... almost two years now (signed up april 2005), and make pretty good use of it. While there's been some odd downtime here and there, its been pretty reliable. Unfortunately I began running into some issues in the last month or so.

On my site I run mainly two main pages, a personal blog (pretty insignificant, maybe a few dozen hits a day), and a game site dedicated to an online RPG 'Ragnarok Online'. The site contains a wordpress log, a wiki, and a database utility so visitors can look up information about things within the game. The database utility is pretty small in terms of traffic, the wiki is the main source of traffic to the page.

Dreamhost contacted me a bit over a week ago to inform me that my wiki was causing too much server load and that they had taken the liberty of dissabling it by renaming the directory, suggesting that I upgrade the software before restoring it. Note that my Disk usage is 3% of max, and bandwidth is sitting at 2%. I presumed that it was because of sql queries, because I was hitting 20000-25000 connections with about 2m-3m queries a day on the wiki database.

At the time this happened, my site became completely inaccessable. A support request revealed that they had set up a throttle on my web page (without telling me), and (woops) didn't set it correctly so it blocked all traffic. So the site was back up after they made the requested adjustments.

My wiki was originally using Mediawiki 1.5, so I decided to update to Mediawiki 1.9. It was a bit of a headache because it needed me to update php, but I got it working. Now the queries is down to about 100k-200k for the same number of connections (Conueries are down from about 2.5MCn to 0.5MCn).

The main issue is the throttle thats placed on my site. Pages keep giving me the error message 'Service Temporarily Unavailable: The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later'. At times its rare that I can even load an entire page without getting multiple broken images. Sometimes pages don't come up at all. I presume that I could just email support and let them know that I've curbed the sql usage and see if they could remove it, but it concerns me that I have an upper limit on how popular my site can be before it basically gets to the point where Dreamhost will render it unusable, especially since this limit doesn't appear to be all that high.

I don't mind updating my package, but the page is just a fansite that gets 60k page requests daily (not unique visitors), its not something I would ever consider getting a dedicated server for. I can't really control how efficient mediawiki is, if the latest version of mediawiki can't run on the server and have people visit it without running into issues, I'd be a bit worried. I'm not certain how I should approach this problem, and am looking for some suggestions about what I can do.

In reply to:
I'd be a bit worried. I'm not certain how I should approach this problem, and am looking for some suggestions about what I can do.

It seems to me that you have made the first initial steps - by dramatically decreasing the number of queries you are using, you have probably gone a long way toward deleting the excessive load on the server.

I'd just respond to tech support with the information you have just provided here, and ask to have the throttle removed. smile

It may be that, as your site continues to grow, you have to do more to optimize the site, but I really can't comment more on that as you didn't provide a url to your site - I'd only be guessing.

You might also consider a more "lightweight" solution than MediaWiki; there a lot of others around that may not be so resource intensive and will work fine for you.

Lot's of things can "run on the server", but not "scale" well for use on a shared host, and the efficiency (quality?) of code between various web application varies greatly (as you have just demonstrated).

Talk to DH about what you have done, monitor your server load from the shell for a bit, and ask them to remove the throttle - they should be happy to do that for you if you have adequately addresssed the problem.

--rlparker

Apparently I had not understood the problem entirely. It seems that none of the Bandwidth, diskspace, or SQL were whats causing the problem. I'm told that my site is hitting the maximum connection limit set in apache causing the server to become unresponsive or crash.

I'm looking into it but I can't seem to figure it out. I don't see any processes that are causing problems nor does it seem to be any specific IP causing heavy usage. My site does get a fair amount of traffic (web statistics says aproximately 60,000 page requests daily), but nothing that stands out as causing problems. Dreamhost tells me that the are unable to remove the throttle because it would cause apache to crash again. They didn't give any real insight to what specifically is the cause, only pointing me to the wiki article on heavy usage. Since they don't give out any limits (instead say that unless you're a 'heavy user' you won't have problems) I don't know where I am in or out of spec.

I'm in the same boat as you... getting 60K hits a day on our site, when all of a sudden I've got 503 errors and Dreamhost says they've put a throttle on. It seems there's nothing to do about it -- they told me they'll remove the throttle when I have less traffic... which isn't really an ideal situation. I don't understand how 60K of traffic daily was fine before, then all of a sudden this month it's been throttled down to less than 25K. By the way, they mentioned that there's a cap of 250 connections per 5 seconds on the server -- and that's for all the users on that server, not just your own.

I don't think there's really anything we can do to try to optimize, seeing as there's absolutely nothing different between now and last month -- aside from maybe DreamHost is putting more accounts on the individual servers now so there are less connections to go around per server? I have no idea. Looks like it's time to start shopping around again..

Edited by mattmayhem on 02/07/07 03:34 PM (server time).

Maybe another site is using up those connections on you.

I'd look to see if you can't get very detailed stats of your site and show that it's not your site that's causing the problems. If there are peaks in your data that don't coincide with the 503's then someone else is causing your trouble and you need the data to call them on it. Say that between 5-6 pm you have huge stats without a lot of 503s yet between 8-9 you have much less volume with many 503s then you may have the smoking gun to point at another user.

Grab all the stats you can!

Wholly