Does anyone use DH as a backup device ?

20GB was great.

But with 200GB, we can really use the DH hosting as an online backup service.
It's more space that i have on my laptop (excluding external hard drive) !

It would be great if i can automatically backup data (encrypted if possible), to recover it of my HD fail or if i lose my laptop.

Does anyone use a backup solution along with their DH hosting ? And wich one ?

I'm currently uploading a copy of all of my digital photos to a nonpublic folder in my account. Besides my physical backups, this is a good way to make sure I won't lose those irreplaceable photos.

The downside? Uploading takes absolutely *forever* when you have tens of gigs of data.

After that's done, I'll upload my digital music. I own it, and I'm not sharing it, so I don't see any legal problems in that. But uploading will take forever three times over, I guess.

I use http://www.2brightsparks.com/syncback to synchronize files accross my lan, that's with the freeware version, but the paid version (25$) has a ftp client that allows you to do the same with remote storage, so I'm seriously thinking of using it, I already had a directory in my dh account with a few gigs of backups, but this allows you to keep them synchronized up to the minute, so it's very handy

http://hentaikid.com - original adult animations
Promo code BATTY 50$ dreamhost discount

That is exactly one of the reasons I signed up for a DH account. I've got ~ 70GB of data (including media files) that I'm looking to backup. The only question I have is whether I can backup music and video files ripped from CDs and DVDs that I've purchased legally. For the time being, until I investigate that a bit more (or hear directly from DH about it), I'm only backing up my other files to my DH account.

My method: I'm running WinXP on my desktop at home. I've installed CygWin, including the ssh and rsync packages. I'm using the following command from within my cygwin shell:
rsync -e ssh av <local_directory_to_be_backed_up> <username>@<host>.dreamhost.com:/<backup_directory_in_DH_account>/

See DH's wiki page on rsync for the details.

My future plans, after I get everything backed up, are to script it to run every night.

The nice thing about rsync is that it only transmits the information that has changed, so after the initial backup of many GBs, the nightly backups will only tranfer the file changes since the previous night.

I'm storing more than a few files...

Though, its not the reason I use Dreamhost. I do actually run a site - a small forum and a getting-bigger gallery~

I've been using my storage space for backups for almost a year. (I use Retrospect, which supports backups to a ftp server.) I was about to remove my earliest backup set so I didn't use up all my space, but now that the limit was raised, I don't have to do that.

I had some problems which turned out to be a bug in my Linksys firewall.

you can use any backup utility - then just schedule a file transfer. If you want encryption or a one touch backup deal... it depends on your platform, but most if not all personal backup solutions should be able to push to an ftp site.

DH Pawns...
29%.... already yeah

Retrospect on Mac or PC? Did it work right out of the box? I just came looking for info on using DH as a backup sources for Restospect. I tried it from my macs but it will not work. Restrospect (6.1 Express) will log in, create the sub directory, create the archive file, then hang and error out with:
Device trouble: “BackupSet”, error 225 (FTP: no write privileges, bad name, or disk full).
I checked the Dantz support site and found the FTP command set Retrospect uses at
http://kb.dantz.com/display/2n/kb/article.asp?aid=5759
I am guessing after USER and MKDIR it is trying to pass some other command that DH FTP does not like.

Yes, currently I backup all my personal data (approx 10GB worth) as well my most commonly used software (20GB) for those times I don't have my cd's/dvd's handy and I'm at a friends house, work, etc.

I use the program from http://www.superflexible.com as it's really easy to setup, has tons of options and works frigign terrific. I have a copy at home to sync my personal stuff and we use it at my full time job to backup our business data to external hard drives. I'm currently trying to talk my boss into getting a DH account so he can do offsite backups of his data as well and use this program. If there is any doubt .. get the 30 day eval version .. it is one of the best kept file sync secrets out there I guarentee it.

Now that the drive space has increased so much ... I'm thinking now is a good time to backup some of my other personal items like my music collection of over 200 GB !!

This was the "full" version of Retrospect on the Mac. It sounds like Retrospect Express also supports Internet backup sets, so that's not the problem. One thing to try is whether you can manually ftp a file to the directory you're using for backups.

I did have one issue, which is that Retrospect only supports active mode FTP. This can be a problem if the machine doing the backup is located behind a firewall. (Active mode FTP requires that the FTP server make a connection back to the machine that started the FTP, which isn't possible if you have a firewall set to block incoming connections.)

It turns out that some firewalls try to handle this. I have a Linksys BEFSX41 and while it worked most of the time, it would fail randomly. I thought maybe DH was limiting FTP uploads or something, but it turned out to be a bug in the Linksys firmware.

The solution was to download an open source FTP proxy called Frox, which can do active-passive conversion. After that everything has worked fine. Every month I rotate backup sets, and I backup ~800MB of data overnight, which takes several hours over my basic DSL connection. During the month I do incremental backups which are pretty fast.

Thanks everyone for your suggestions.

But no one use a solution with encryption ?

I would like to use a software like Mozy (easy to manage wich folder i want to save, what kind of files, ect...), with dreamhost.

I've started to copy some of my files to my hosting space, mostly digital photos and such. But it sucks that I only get an upload speed of 38KB/s..... that's just slooow. So it takes like 3 days to upload 10GB of photos.
Using DH for backup may be good if you're in USA and the network is fast, but from overseas it doesn't seem that useful.

upload speed depend more from your line than where you are.

I can upload at 1Mbps. That's slow for the first time, but after the initial backup, all following incremental backup should nt take that long.

I live in Denmark, and my upload speed on my line is 1mbps, which I can take full advantage of.. I use my space for backups of all my digital fotos (not accessible from outside), and now, since everyone has a minimum of 200GB storage space, my bootlegs in FLAC format, which are of course legal in both the US and Denmark. So far it's been going for three days, almost half way. :)

Why take all the trouble to backup online and waste bandwidth when a extra harddisk is so cheap nowadays??

Even portable harddisk is affordable nowadays. I really see no reason to backup online unless you always moving around the country and always need to data online.

Yeah, I agree. Just the time uploading files...what a pain that would be.

I guess, if you need an external backup (offsite,) dreamhost would work...but seems easier to purchase an external drive and leave it in the trunk of your car (unless you live in Arizona.)

The reason to backup to dreamhost is that they are "offsite" for pretty much everyone. I live in the eastern US, so it's great to have a hard drive maintained for me in California. If something calamitous happens here, my local external hard drive may also be lost/destroyed along with my computer. But the California location would probably survive whatever event wiped out both my laptop and my external backup drive here.

Don't get me wrong -- I keep those local backups too. But it's the diversity and multiplicity of a far-offsite backup that makes DH attractive for this purpose.

Ah, that makes sense and is likely what is happening to me. I confirmed I can FTP to DH via the OSX Terminal but had not thought of Active/Passive FTP. I will see if my Firewall has capacity for handling Active FTP and if not will try out the proxy you suggested. Thank you so much for pointing me toward this :-)

Hey Guy,

As some of you can backup files at a super fast 1mb, but have you test and download those files to see how fast it goes? I'm uploading around 800-900 kb/s and testing the download speed only reach 60 kb/s. It is so slow to download the file then uploading it. So for example that a backup data is 10 Gb take fews hours to upload but take days to download. Damn that sux :P

Cheers,

Does anyone know if there are file size limits on DH? In other words, if I'm patient enough, can I upload a 40G file, or would it have to be broken up into smaller chunks (like 4G)?

Bob

In reply to:

---

I'm uploading around 800-900 kb/s and testing the download speed only reach 60 kb/s.

---

I always get something close to 150KB/s when downloading from Dreamhost, but that is the limit of my internet connection.

In case anyone else wants to know, here's the response from DH support:

"While we don't have any inherent filesize limits, I think it would be
practically better to break your files into smaller, less than 2GB
pieces. This is because our FTP servers do not support upload resuming,
so you would best be off uploading your data in small pieces."

Bob

I am now using DH as my offsite backup, including encryption.

This is a bit long, but here is what I am doing.

Our home LAN consists of 4 PCs (my Laptop, my wife's laptop, my daughters' computer and my MAME machine). I also have 3 special purpose servers: (1 IPCop firewall, 1 Linux box running Firefly media server and 1 XP Pro box that has been hacked to support RAID 5 and is the main file server for the house).

The only place that there is a need for automated backups is of the 2 laptops and here is what I have done:

The XP Pro "server" is running TrueCrypt and is sharing out a 50GB volume to the LAN.

Also running on the server is DeltaCopy Server (a freeware rSync implementation for backup on Windows). Each night (in the middle of the night) the DeltaCopy client on each laptop syncs their data to a folder on the encrypted volume. This takes only a couple of minutes over the LAN since most of the data has not changed. Currently we are using about 19GB of the 50GB TrueCrypt volume.

Once a week I have a script that does the following.

1. Dismount the TrueCrypt volume
2. Use a windows version of the unix Split command to chunk the TrueCrypt volume file into 250MB chunks. (200 chunks total).
3. Re-mount the TrueCrypt volume... putting this step in here minimizes the amount of time that the LAN backup target is unavailable
4. runs rsync on the PC (cwRsync) to rsync the chunked up TrueCrypt volume to my space at DreamHost.
5. Deletes the local copy of the chunks.

The whole script takes about 4-5 hours to run, though the impact of having the local data backup unavailable to the LAN is only about 30-40 minutes.

Much of the time is just spent in calculating the block hashes by rsync and not in actual data transmission, but when you are dealing with 50GB of data it still takes a long time.

The inital upload of the 50GB of data took about 4 days, but the rsync of changes taking 3-4 hours is perfectly acceptable to me. If I can get it running smoothly enough, I may schedule it to automatically run each day while we are at work.

I have not tested the restore of the data - I have to download all 50GB in order to re-create the viable TrueCrypt volume, but I plan on doing that soon.

Some additional info:

1. I have Verizon FiOS service 15Mbps download/2Mbps upload
2. I am not terribly concerned with the security of the truecrypt volume on the local server so I leave it mounted all the time. The security of the encryption here is just so that someone at dreamhost (or a hacker who might break into dreamhost's servers) cannot peruse my data.
3. It is a potential security threat to have 2 copies of the same TrueCrypt volume with different states of data in them - potentially there can be vectors of attack that depend on differential analysis between the two volumes. That is why I delete the local copy of the rsynced chunks as soon as I am done with them.

Again, I am not overly concerned with high end secutiry from TrueCrypt, I'm mostly looking to keep prying eyes out of my data when I back it up offsite.

Ted

Ted,

I am trying to use DeltaCopy to sync data directly from a Windows XP (Home) machine to the backup account, but my inexperience with a Linux environment is hampering my ability to troubleshoot....

EDIT: I have merged several cross-posts into a new post:

http://discussion.dreamhost.com/showflat.pl?Cat=0&Board=forum_troubleshooting&Number=112802&page=0&view=collapsed&sb=5&o=14&part=Edited by quena on 11/18/08 01:07 PM (server time).