I'm a bit surprised that nobody has posted some real-world numbers about legitimate data.
I'm a part-time photographer with a RAW photo collection that extends back three years. In 2007 I shot over 5,000 photos (being in school for my photo degree helped) and for that year I have about 30 gigs of RAW images, JPGs, TIFFs, and other assorted image files. I shoot with an archaic Canon 10D (6.3 megapixel) that averages about 6 megabytes per image with Adobe's DNG compression. On top of these "measly" 6 megabyte files, I have Photoshop layered PSDs and TIFF files which are at least 36 megs and one of which is over 1.6 gigs. Now, I don't photoshop every image - many are edited out and I keep them for other uses (data hoarding). All in all I have 260 gigs of still image files, 90% of that within the past three years.

Still images pale in comparison with the amount of space needed for video - particularly HD video. Standard def video takes 13 gigs an hour depending on your compression algorithm and only goes uphill from there.

I'm looking for the best scalable option to load my eggs in one basket for management ease. So far the best I've found researching is RAIF. Haven't played with it yet, but I'm hoping it fills the gap between ridiculously expensive hardware and the "it's full, now what?" problem of most NAS solutions.

I back everything up to Univac punchcards...keep them in a dry place away from the kids and those things never corrupt.

For all you guys who use external hard drives and store them elsewhere, are the drives encrypted and password protected in case they get stolen?

Better yet, when you burn DVD's with sensitive data on them, do you encrypt that?

It seems like a pretty big security risk to have gigabytes of personal data just waiting for someone to take off your hands.

Y'all might want to check into crashplan/crashplan pro (or crashplan pro for business, if you are a business).

The clever idea is that you (and a friend or two) all install the software and back up to each other's external drives. (the contents are encrypted before being sent over the internet.) Thus you get an off-site backup, but you also know exactly where your off-site backup(s) are.

It is pretty inexpensive, at around $25 for the basic version, and $60 for a version that will preserve multiple versions of a file, etc.

www.crashplan.com

I use an online backup service at https://mozy.com/
It's actually pretty nice. Only $55 per year for "unlimited" backup storage. I put unlimited in quotes because I'm sure if you threw enough terabytes at them they would complain, but I have over 80 GB of files backed up there, and I haven't heard a peep out of them.
If you have only 2 GB of data or less then the service is free.
Your files are encrypted, and you can supply your own encryption key if you want. They keep multiple versions of your files for a month, so if you need that PROJECT2A.XLS file as it was 5 days ago, and as it was 2 weeks ago you can do that. If you need your files you can access them from anywhere through their website, or you can get them to burn some DVDs and mail it to you.
Of course since you're backing up online you need a decent ISP that won't complain if you're uploading. A faster connection helps, but they cap your upload speed to 1 Mbps, so if you have a lot of data it'll take quite a while to do your initial backup. After your initial backup they only do differential backups which is much faster. My first backp took almost two weeks. ick.
Plus, the folks that run the company have a sense of humor not unlike the folks here at TR. ;-)
https://mozy.com/blog/2006/09/11/probability-is-fun
https://mozy.com/blog/2006/07/06/chinese-bluegenes

I posed the same question in an article I wrote a couple months ago with the perspective of making data survive generations:
http://jonac.wordpress.com/2008/03/04/the-long-term-data-storage-co...

I don't really have an answer but until something that solves data longevity in the long-term and is permanent is invented I'm going to have to port all my data over from one set of hard drives to another every 5 or so years. That's about as long as I'm going to trust a drive for stability and integrity of my data. Maybe even go from one RAID1 system to another. What a hassle with 2TB of data.

Personally I've been contemplating dropping down a wad of cash on the Drobo (just for the ease of use of the damn thing). I mean heck when you consider all the alternatives, the Drobo is the hippy, flashy Apple-style-feel-good alternative for data storage.

Well I can tell you one thing dont try backing up anything with a western digital My WorldBook NAS drive what a total POS. I picked one up today thinking it would be a nice way to share files etc boy was I wrong. It will stay connected for maybe 5 minutes and then the conection dies. If you try to access it from more than one computer it dies the software is flaky as hell on top of that. I like western digital hard drive and have probably 8 of them but the My WorldBook drive is complete and total JUNK Stay away from it.

Easy:
WIndows Home Server. Easy for Joe Six Pack to use. You can even designate an external drive for WHS to backup onto and then you can take that device offsite.

I myself have ~1.6TB's of data on drives that are mostly full. On top of that, I migrate stuff off the collection every few months. After like a decade, I've got around 500 fully stuffed DVD's sitting on shelves, and the number keeps growing. I don't need to hear anything else about BluRay until they produce burners under $200CDN.
Edit: I've also found that for me it's faster & cheaper to use DVD5's and not DVD9's.

http://www.pcworld.com/article/148765/backuputilities.html?tk=rss_r...

Might be useful.

Home: Vista's built-in backup to 1 tb drive.
Work: Workstations and servers Symantec Livestate recovery to NAS, then offsite to hosted datacenter.

3 or more BIG hard drives (most expensive part)
cheap SATA RAID5 card ($40)
Old computer (Older celeron, lower power - put in the basement)
Compact flash to IDE adapter, small CF card ($10)
FreeNAS (FREE)

Gives you stable RAID5 fault tolerant goodness and can share the files to just about anything. ZERO Linux knowledge required.

http://www.freenas.org/

When you get into Terabytes of data, buisnesses use something like a LTO tape backup drive. If you wanna get serious about REALLY backing stuff up, you spend the money on a tape solution. The reason for this is lightning, flooding, theft, fire, etc. etc.

I think it's a timely question, since backup strategies for home users have become more difficult than ever.

I beg the differ. Back-up strategies are far more easier and affordable than ever before. It just a recent event where mainstream users could do seamless data back-ups.

It used not long ago where you had to rely on expensive, propertary tapes or far more fragile but cheaper media like floppies and optical media (CD-R/RWs, DVD +/- R/RW) for data back-ups.

External HDDs, their interfaces (USB, FW, eSATA), applications (WHS, FreeNAS, and other solutions) have matured enough that even a computer newbie can effortless back-up their data without breaking the bank.

I think the real problem is it is harder then before for data rat-packs (not mainstream users) to do back-ups with 1TB+ collections of pr0n, wares and other illicit content. ;)

Edit: I forgot about USB thumb drives. They are an excellent back-up storage for lighter, but mission critical stuff like (taxes, documents, resumes etc).

I've got a Thecus N5200 NAS connected to my router over gigabit. I'm using 5x 750gb drives in a RAID 5. Any machine in my apartment can access whatever it needs to pretty quickly. I can even play music and movies directly to my PS3 from the NAS.

I don't use all that backup software, it require you to be organised yourself, which I'm not really, and if you are not it would just end up backing up useless junk since it would have no real idea what to sync and what not without my input and constant attention

This is where a SSD would be ideal... DVDs and CDs don't have the longevity required for archival backups. SSDs approach reasonable lifespans until better technology comes out.

When it comes to film pictures, even they have storage issues with fading dyes, but they don't go completely away all at once. Of course if you have a fire, everything is toast, but the same is true for digital.

Probably the best way is to set up a very basic server off-site with USB-based SSDs and move your file diffs over broadband. the first copy could be local before moving it off-site.

Too bad SSDs are currently fairly pricey.

This is more practical and likely more reliable than a lot of solutions.

-Mark

Backups are worthless without verifying the data. This is one point no one seems to be addressing. Is there any easy/automatic way to verify what you have backed up is exactly the same as the master?

At the moment I uses Md5Sum hashes for more important files. I manually check them, but it takes forever.

if i know everyone here, 90% is pr0n. You dont need to back it up. as for backup, i use BR-RE. 25gig is more than enough. also back up certain things to google, and i run a raid matrix.

I am sure some of us would like to hear the recommendations you pass on to your friend.

How about a TR definitive backup guide? Covering Joe user through to advanced backup needs. You could cover the hardware and software requirements of each implementation. I wish there was a Wiki about this.

I think something on security should also be mentioned.

autobackup to a eSATA WD 1TB GP drive in a AZIO usb/eSATA case.

Whenever it fills up I delete it and start again.

I just use two external 500 GB USB 2.0 external hard drives. I have a spare internal SATA 500 GB sitting in a box, but haven't needed it yet. ( :
I burn 4.7 GB DVDs as well, but i keep the most important data on the external drives.

Am i crazy, or shouldn't external hard drives last longer than internal hard drives when the PC is left on all the time and the externals are only powered on for a couple hours every month?
(Assuming the externals are always handled very carefully, never bumped or dropped, etc)

Currently waiting for Sandisk to make a 1TB USB x.x thumbdrive type thing. Until then DVDs'll do.

HP Homeserver. Already saved my bacon on my work laptop with a complete restore. Also have the Jungle disk plugin to backup my photos and docs from the homeserver to S3 for off site.

Can't recommend it enough for any windows user.

In a situation like this you have to take out requirement #5 to have a chance at reasonably achieving the goals. Without even being able to purchase any kind of hardware there isn't any room for good solutions. With that requirement the only option left is backup to a ton of DVDs which doesn't take much technical knowledge and only requires the purchase of DVD-Rs, chances are he already has a DVD writer.

Take out requirement #5, or at least take out the 'purchase' part, and a situtaion like this calls for a pre-built NAS or something along those lines. A slightly more technical-minded user could go with a WHS box, or for a technical user with time to put in to it a homebrew server on Linux. An in between solution is unRAID http://lime-technology.com/

I use my PC mostly for making music using cubase, Recently I upgraded to a new version and wanted to clean up some stuff on my audio drive, so I Rar'd it up in 700MB chunks and burned them off to a few DVDs.

Then after formatting my drives, reinstalling windows and all of my apps (anyone who has an extensive collections of VST effect or VST instruments will know what a time consuming labourious job this is) and then I get to the backup of my audio drive...

copy over the rar files from the dvds, and bang one file wouldn't read from the brand new DVD-R resulting in about 8 months of lost hard work!

The moral of the story? Cheap DVDs are not a good place to put your important backups!

I am now considering investing in something a bit better, maybe having some NAS on an old cheap pc and backing stuff up to there every few weeks, but I sure as hell wont be putting 8 months of my life on a DVD again.

Time Machine + an external USB drive that's plugged in at all times, plus Carbon Copy Cloner and another external USB drive that has a 1-to-1 mirror of my stuff that gets made every week.

I can't imagine how you could have more than 500gb of legit data and games.

10 160gb hard drives in a RAID5 array on ubuntu for me..

plus i have the music mirrored on another drive in my HTPC, and anything really important i e-mail to myself on gmail in an encrypted format :)

If you have the time and good budget, the best solution is a iscsi san with about 5 1 TB disks. A few nights with LVM and linux software raid, and you'll never go back to storage appliances again. I suggest openfiler for the novice however after learning all the internals you can throw away the clunky interface and just use command line.

First off, I don't have irreplaceable data, but I hate having to replace data that I've accumulated and thought was worthy of keeping.

My main computer:
Q6600
2x 150GB Raptors One has windows + Application installs the other has the game installs

4x 750GB Seagate One has Movies one has Game ISO's(legal backups) one has Photo's and Music and the other has the various programs and applications I've downloaded and use for various tasks

Now my home file server has a lot less storage space... only 2x 750Gb Seagates that, every so often, I backup the Program saves, game saves, Photos and other little bits of data to. the rest of the stuff is easily replaceable. I use my file server for a lot more than holding backed up data though, the 36Gb Raptor and 120Gb Maxtor holds WinXP + a few games so my buddies who come over can play DoW or some other game with me.

My #1 recomendation though is for people to dump some cash on a 1Tb or 2 Tb drive and every 2 weeks back up all of the irreplaceable data like Photos, personal data, and maby music. Screw the DVD's...

Most people will say that my data isn't really secure and backed up, I don't really care because I've never had a HDD fail on me, hell I've never kept one long enough to fail on me... I do plan on having the 750Gb's for a while but liek I said if one fails the data on it is backed up and/or easily replaceable.

I really see no need in having some exotic back up sloution that is inefiecient compared to a simpler solution that will work just as good.