Are there any limitations on MySQL memory usage?

Hello there,

I've been looking for a new host for a vbulletin discussion board after suffering all kinds of shortcomings from many web hosts.

I'm interested in moving to DreamHost, but I have some concerns about the ability to backup the forum DB. Some hosts I've been at impose memory limitations on MySQL operations that cause the backup process to fail or result in a incomplete backup (There is no simple way to verify the backup file).

Are there such limitations at DreamHost? What is your experience with data backup?

Best regards,

Edited by AntiVirus on 02/07/08 00:14 AM (server time).

Well, besides the manual backups that we take (via the web front-end of our apps), DH support has an automated backup system that mostly works great.

If you know the specific memory requirements of the backup process or the tablesizes, you can probably contact pre-sales support via the following contact form:
http://dreamhost.com/contact.cgi

What are 50DISK50, 3DOM50, and 1IP1DOM50?
They're Dreamhost coupons!

Thanx for the info.

I checked the wiki and it seems the automated backups are for the hosted files only, unless the database files are somehow included in those file images, which I couldn't verify.

Gonna try the contact form then. Thanx again :)

In reply to:
I checked the wiki and it seems the automated backups are for the hosted files only, unless the database files are somehow included in those file images, which I couldn't verify.

Hmm, I guess we should update that page, then. Which page was it?

Through the panel, you can restore from any of the five most recent automated backups taken daily for each separate database in your mySQL server.

What are 50DISK50, 3DOM50, and 1IP1DOM50?
They're Dreamhost coupons!

In case it helps I had a problem with a postnuke plugin that ended up constantly querying 50000 rows (not my intentions!) So if your board gets huge, you might have to upgrade to a Private Server instead of normal hosting.

The current SQL backups are rather good, and you can always do your own backups to complement them.

Tech Rant
Use promo code 2DOMAINS4LIFE for $50 discount and, well you guessed it two lifetime domain registrations!

I would not recommend DreamHost for your forum. I constantly have MySQL failures, server errors, and just straight up down time like right now. My site is down again.

Here is a response about the back ups I recieved .

Reply from DreamHost (Jan 30th, 2008 - 14:27:03 / #16085243)
Subject: RE: MySQL Restore not working
Hello,

> im in the process of restoring from the backup I made, but there are items in
> it from the hacker that need to be removed and I dont see where they are. So
> I
> wanted an older back up to restore from.
> My issue is that your backup is not working fine and if I didnt do a backup
> manually on my own your "Restore DB" function would of bascially erased my
> database, and according to you, not had another way to get it back.
>
> my databse is over 200mb and has been up and running for a few years. What
> happens if there is a system wide failure and your team needs to restore the
> entire server? Would you be telling your customers sorry because the
> databases
> did not back up properly?

Unfortunately, there aren't any backups available. Nightly backups aren't
being ran. The machine that your database is hosted on needs a memory
upgrade. This will be performed in the very immediate future. We
sincerely apologize for any inconvenience this may have caused. If you
have any additional questions, please let us know.

Thanks!
Brian S.

In reply to:
I constantly have MySQL failures, server errors, and just straight up down time like right now. My site is down again.

Well, a *lot* of sites are "down" right now due to a scheduled, and announced, major move of a whole cluster. Could that be affecting you?

It's unfortunate that you have had bad uptime experience; mine has actually been very good in spite of some recent hardware issues. I guess as with many other things, YMMV when it comes to "uptime". wink

I can't help but be a little surprised though that you have a forum "up and running for a few years" and found yourself "hacked" without decent/current backups of your data. shocked

DreamHost has been very up front in their TOS, and their documentation, that any "backups" they do are a "courtesy" and are *not* GUARANTEED to be available. Even if they represented otherwise, which they don't, common sense should prevail when it comes to backing up data that is stored on someone else's machine.

In reply to:
"My issue is that your backup is not working fine and if I didnt do a backup manually on my own your "Restore DB" function would of bascially erased my database, and according to you, not had another way to get it back.

It sounds to me like that might have indeed been the case; were you expecting something else? You should rely upon *your* backups. It's great that most of the time the DreamHost backups are there for us, but that is *not* something you should count on always being the case.

In reply to:
"What happens if there is a system wide failure and your team needs to restore the entire server? Would you be telling your customers sorry because the databases did not back up properly?

I'd certainly hope they told us that if that is what happened. The truth is always best. Of course, DreamHost tech support folks are kind and compassionate, so it is totally in character for them to apologize for your not having something available that they never promised, and for them to commiserate with you. They could have just as easily said that backups are made by them when and if possible, but that you are responsible for your own data, and have pointed you to the TOS. wink

Hey, I know it's rough to get "hacked". Did you not keep your forum software updated any better than you kept your data backed up?

I can empathize with your situation, and I'd like to have sympathy for you as well ... but I can't. Running and maintaining a website is not an "auto-pilot" kind of thing; you have to take reasonable steps to protect yourself and your data.

While I don't know what you did, or didn't, do to secure your site, you have told me what you didn't do to protect your data - you didn't keep it backed up properly ( rotated and aged) or you would not have been disappointed at all that DreamHost didn't have a backup that you can use instead of your own (since you can't vouch for the integrity of your own backup).

It's a hard lesson to learn, but maybe your posting here will serve as an object lesson to others. wink

--rlparker

Just to clarify something with the mysql backups that DH does. I understand they are not guaranteed and are a courtesy to us, but when I did speak with them when the site got hacked and I was hoping they had a back up to save me, they suggested using the restore feature. So I took their advice and used the restore feature and come to find out the back ups that were made contained no data at all in them, causing my current database to be replaced with a empty one with no way to return to the previous full one. I had made a backup so I was able to restore it that way after many hours. I contacted them after the empty database over wrote my database and that is when they told me that the backups had not been working. I just wished they had checked before hand to make sure it was there instead of having to find out after the fact. Again I understand it is a courtesy but there has to be some type of accountibility with this stuff. I feel that if you are going to offer this, but then in the same breathe put in the TOS it probably wont work, then why offer it? Not everyone that has a site is a super admin and they may entrust that DH and its services are what they say they are.

I agree that we should of taken better measures to ensure we had backups and a more secure site. Live and learn! But the issues we have had with DH have a long history that do not have anything to do with the hacking incident, about 22 pages worth.

Just like anything else some people will never have a problem with DH and there hosting and some people like us will.

DH has had its fair share of outages across the boards that have been very public. I mean they have their own outage page! http://www.dreamhoststatus.com
I would direct any new user to take a look at this page, do research on any host you are thinking of going to and make a smart decision.

Edited by nepcw on 02/09/08 07:30 AM (server time).

Oh , and also think for the future. If you think your site is going to grow and grow then start off with more than you need. Moving a site can be very time consuming and difficult once it gets big.

Shared Hosting is great cost wise, but it can be outgrown very fast.

ok one more thing to add. Right now we are experiencing 500 errors all over the place and when we inquired about it we got that email that was posted above.
Then a few days later we get an email from a different tech telling us our server needs more memory and it will be upgraded soon. So we are confused whether we need to move to a bigger server or if we will be ok for a little bit.

There is an Edit button.

I really do understand your frustration, and you make some valid points about depending upon something that may or may not be there when you need it. wink

In reply to:
So I took their advice and used the restore feature and come to find out the back ups that were made contained no data at all in them, causing my current database to be replaced with a empty one with no way to return to the previous full one.

Just to be fair, you should note that you could have prevented that from happening by selecting the options in the Restore DB form to make tables with alternate prefixes and *not* overwrite your data until you could see that what was in the backup was useful.

Yeah, that would have been an "extra step", but there is a reason you have those choices offered.

--rlparker

Did these 500 problems start immediately after you were "hacked" or after you recovered from the "hack"?

we had them before and after the hack. After the hack it was def worse for awhile. Then the last week we have been really good, with a heavier volume of traffic.
We have had issues with mysql and site outages almost the entire time we have been using DH.
If you would like me to post the 22 some odd pages of our support history i would be glad to.
and if I miss anything I will be sure to use the "Edit" button as not to offend anyone

I made a manual backup before I used the restore just in case something like this did happen. I didnt expect the database to be replaced by a completely empty one. So I did take an "Extra Step" by manually backing it up first. Now that I used the restore feature I'm so very confident the other options they give would of worked perfectly..... ;)

I'm taking responsibility for the hack and not doing backups regularly, there is no question there.

Like I said before though, our support history is long with them, way before this particular incident.

In reply to:
I made a manual backup before I used the restore just in case something like this did happen. I didnt expect the database to be replaced by a completely empty one.

Whew, thanks for mentioning this. I was otherwise going to have to chide you for doing an overwriting restore of an unconfirmed backup before taking a backup of the original data. :)

Looks to me like you know what you're doing and are just trying to advise the OP to make sure they do their own backups and to confirm their "goodness", right?

What are 50DISK50, 3DOM50, and 1IP1DOM50?
They're Dreamhost coupons!

you got it! :)

Are there any limitations on MySQL memory usage?