Fear the Cowboy

Life of Microsoft Open Source Developer

RAID 5: These aren't the droids you're looking for.

clock August 11, 2008 14:49 by author Garrett Serack

My earlier post on Windows Home Server's Drive Extender vs RAIDa lot of what I said was a good example of somethin' my pappy once told me:

"Good judgment comes from experience, and a whole lotta that comes from bad judgment."

Well, rest assured that I've had my share of bad judgement, as I'm sure others have, given some of the feedback I've received.  Charlie pointed out to me some of the other really good reasons why you're better off not going with RAID5, rather than just using Drive Extender:

While RAID5 hardware is fast, it is also pretty much universally incompatible. In other words, if your RAID5 controller fails you will have to find an IDENTICAL controller to replace it or your array will be unreadable.
For those playing along at home, this is bad. If you think your data is valuable enough to keep around, and you're worried enough about failure that you're going to do something about it, think about what else will fail--not can, but will-- hard drives, controller cards, motherboards, ram, network adapters, power failure.  I'd mention the possibility of meteor strikes, but ... I'm gonna play the odds with that one.

When I think about the how each of those failing is going to affect my ability to even recover data, the controller card is a nasty wildcard. There is no standard layout for how each controller card uses all that storage, and it's going to be needing replacement with the exact same controller.

All drives in a RAID5 array have to be the same size.
Hey sure, no problem. I can get 750 gb drives the same size in the future... Uh, except there is no guarantee that the drives will be exactly the same size, (even if they are labeled the same), and it's possible that other changes in the future could render them so obsolete, that it could be hard to even source drives of a particular size.

Furthermore, as you progress into the future, with Drive Extender, you can simply buy any size of drive that fits your budget, and add it to the server. No worries about how big, what brand, how fast, etc.

Drive Extender gives you the flexibility to NOT duplicate files that don’t need to be duplicated (making it MORE efficient than RAID 5 in some cases).
And this, is a really really good point too. Sometimes, I want to have data that I really don't care as much about, or I'm less worried about it being backed up and online.  With Drive Extender, you can mark some content as not requiring duplication. Like maybe TV shows I've recorded on Media Center. Or those videos of my in-law's trip to Cleavland. You know what I mean.

RAID5 rebuild times with a single drive failure are often as long as your 3.5 month ordeal of recovery!
Well that's certainly true--I took all summer to restore my data, and I had no idea until I got it back if I would.

RAID deals at the block level and thus knows nothing about the interesting file information (metadata, etc…).  Thus it will NEVER be able to be smart about your storage.  But Drive Extender is file based and tons of innovation can happen (will happen!) leveraging this fact.
This is the point that I think is even more important, and yet so many folks will either ignore or not really understand.  RAID is a system of aggregating independent disks and presenting that larger volume that to the OS as a single large drive. It works at a block level, meaning that your RAID system really knows nothing about files, just blocks. And because it's unaware of that, there is so many optimizations that can't be done with it. With file-level data redundancy, the system knows a lot more about it, and can begin to make decisions that would be much more beneficial.  Think, down the line in a couple years, as I add a few more hard drives to my growing server--which already contains drives that are connected by Firewire, SATA, ATA and USB (*sigh*)--it's possible that the technology could realize that I rarely ever touch some kinds of data (or specifically, some types of data) and it could migrate those files to the slower media, and keep the faster media for things that need it.

Now, my momma said that I should always end by sayin' somthin' nice, so I'll leave ya with this:

So is there anything good about RAID5 or are you just some sort of once-scorned-RAID-hater?
I've said it before but it bears repeatin': RAID--in general--is great for performance, and RAID5 adds to that, availability.  My desktop at home has 1.4 terabytes of space, striped (that's RAID1). My desktop at work has 2.8 terabytes of storage, again, striped. Sure, sure, with 4 750 gb drives, and using striped storage, I'm subject to failure at 4x the rate I was before (probably more, cowboy math ain't that great... :D). On those systems, I'm not leaving anything there that I can't afford to lose, either because I've backed it up, or it's recreate-able.

 

But WHOOOOOSH, it sure goes fast!




Windows Home Server's Drive Extender vs RAID

clock August 11, 2008 10:02 by author Garrett Serack

I use Windows Home Server at home to store *everything*... it's really quite a fantastic product. It has a feature called Drive Extender, for which Wikipedia describes nicely:

Windows Home Server Drive Extender is a file-based replication system that provides three key capabilities:[12]

  • Multi-disk redundancy so that if any given disk fails, data is not lost
  • Arbitrary storage expansion by supporting any type of hard disk drive (Serial ATA, USB, FireWire etc.) in any mixture and capacity
  • A single folder namespace (no drive letters)

Users (specifically those who configure a family's home server) deal with storage at two levels: Shared Folders and Disks. The only concepts relevant regarding disks is whether they have been "added" to the home server's storage pool or not and whether the disk appears healthy to the system or not.
Shared Folders have a name, a description, permissions, and a flag indicating whether duplication (redundancy) is on or off for that folder.

If duplication is on for a Shared Folder (which is the default on multi-disk Home Server systems and not applicable to single disk systems) then the files in that Shared Folder are duplicated and the effective storage capacity is halved. However, in situations where a user may not want data duplicated (e.g. TV shows that have been archived to a Windows Home Server from a system running Windows Media Center), Drive Extender provides the capability to not duplicate such files if the server is short on capacity or manually mark a complete content store as not for duplication.

Here at Microsoft, we have an internal mailing list for WHS, and every once in a while, someone asks one of the following questions:

Isn't RAID better than Drive Extender?
Why should I use Drive Extender instead of RAID?
Which RAID card should I buy?
How good is software RAID5?

I try to ignore those threads, but when the responses start coming in about the merits of RAID vs simply using DE, I end up getting itchy, and chime in. The topic came up again, this last weekend, and I recycled an old response, and it started looking like a good blog post... so here's the skinny.

First, the reason why you don’t want Software RAID 5

First, there’s a big gap between software RAID5 and hardware RAID5. Software RAID5 is slow. Damn Slow. Faster than that… maybe pretty damn slow. Not a great solution. You won’t be happy at the end of the day (see section below “Why you don’t want RAID 5.”)

Hardware RAID5 is fast. Zippity fast. So is how fast you will lose your data.

Why you don’t want RAID 5

RAID 5 is not about data integrity… it’s about performance and availability

If you want your data to be safe, replicate it. Back it up. Put it in more than one place at a time.

If you use RAID5, you still need to back up your data.  RAID5 is designed so that a single drive failure, will preserve your data, and make it available (but slower) until you get another drive in place, when it will rebuild the missing volume.

Here’s the kicker. What happens when a drive fails, and you are not there? If the system is in use, it’s going to get really really busy, and all of the drives in the array are going to get a lot of use.

When that hard drive fails (and you are not planning for IF but WHEN), and the others pick up the slack, the chances of losing a second drive go thru the roof. What will you lose if a second drive goes?

This is common, especially in a server/computer in a home environment, where the drives may not be busy most of the time.

One other contributing factor to multiple drive failure in RAID5, is people tend to use the same brand of drives, especially if they are the same batch (ie, you bought them at the same time).

My personal experience with RAID5

I had a server running RAID5 at home, it ran perfect for over a year (actually, close to two). One night after I went to bed, a drive failed. 3 minutes later another failed. This was a 2 terabyte RAID array.

I came down in the morning to my worst nightmare. Every bit of ‘valuable data’ I had in the world was now gone. In desperation I scoured the internet, and finally found a piece of software that (for $40!) could  recreate every file that I still had data for, if not a little slowly. I rushed out and bought 3 750gb drives, and started to restore everything I had lost. The restore process took 3 and half MONTHS, running full time, around the clock. The good news is that I was able to get one of the failed drives spinning again, and I lost a total of one file.

What did I learn?

RAID5 doesn't back up my data. Sadly, I thought it was safer. Worse than that, it was actually less safe. A single drive failure would have meant nothing. Add another drive, and keep chugging. Potentially, it may have taken a few hours to rebuild the lost volume, but I could have been using it while it did.

A second drive failure would have meant I was offlined for the time it took to restore--If I actually had a backup. Still, not bad, considering that would have been less than the 3.5 months.

But a two drive failure(which is fairly likely)--without a backup-- is a nightmare.

If you value your data, replicate. I now have a home server with 6 250gb drives and 3 750gb drives, and the data that I value is replicated. (and the really valuable data is foldershare’d to a friend’s house, and vice versa, giving us offsite backups too). Sure, it’s not as ‘space efficient’ but at least I can deal with a drive failure.

Raid 1 (mirroring) is the only RAID where a failure doesn’t increase drive activity drastically—well, reads are all going to one drive now, but if you had 8 drives mirrored in 4 sets, typical access won’t cause all the drives to get busier.

Raid 0 of course, is purely about speed. Half the safety at twice the speed. That’s what I use in my desktops. (where I want it fast. I of course back up anything that I’m not willing to lose to the server.)

What is my Advice?

Know this: if you are using hard drives, one day, you will experience a drive failure. Not 'might', but 'will.
How you are affected depends on your choices.

Determine how valuable your data is.
Stop thinking about the price of the hard drives. Disk Space is very cheap. It got cheaper while you were reading this. It's the stuff you store that's not.

Are you planning for the inevitable, or playing the odds?
I can talk all day why DE is better than RAID, or why one particular strategy is better than the other. At the end of the discussion, you're still the one making your decision, and you're probably pretty smart. (You're reading my blog). Ask yourself: why are you doing what you are doing?

I'll leave the rest to your imagination.





The Cowboy

What I'm Tweetering about...

 

follow me on Twitter

Calendar

<<  September 2010  >>
MoTuWeThFrSaSu
303112345
6789101112
13141516171819
20212223242526
27282930123
45678910

View posts in large calendar

Sign in