Once and for all - RAID5 implementation?

[oDii]

I posted a while back some basic plans to build a file server, and my intentions have since firmed up somewhat [details aren't important]. What I want to finalize however, is how I'm going to implement RAID5. There are three options for me as I see it (corresponding to the poll):

• Software RAID
• Onboard RAID
• Dedicated RAID card (will be cheap)

I've heard so many positives and negatives between the 3 and I'm just utterly unwanting to make the decision based on my limited knowledge. The server itself is going to be used as a general purpose home file server - backups, filesharing, whatever. The machine will likely be running Debian (as I'm used to it) and SAMBA, an E6300 (so software RAID isn't as much of a performance negative; will be doing other things with it don't worry) and will have enough RAM. The thing I'm looking for is ultimately the least trouble from the machine - I don't want to babysit it - and if something goes wrong I don't want to find myself with a dead array because the controller is being stupid. This is why I'm perhaps ever-so-slightly leaning towards Software RAID as despite the performance negatives I've been told it's perhaps the most reliable and easy to work with once you've got your head around it (and can be moved between machines). I've heard bad things about onboard RAID (particularly NVIDIA's rather inexact implementation) and don't want to be SOL if I want to upgrade the motherboard later on.

A dedicated RAID card I'm unsure of as I've heard mixed results with it - while it would allow me to move it between machines with the least fuss (theoretically), cheap cards (ie. ones that I'll be getting as server bound components are damn expensive) supposedly shove all the work onto a CPU so in effect I've paid far too much for a software RAID setup.

Summary: Fileserver - a RAID5 array I can transfer between machines if required, extra work for the CPU isn't a problem but dedicated RAID cards are (availability and cost in New Zealand --). What would you recommend? Cheers guys.

 

[DougLite]

Fast. Cheap. Reliable. Pick 2.

There is no "once and for all" answer to questions about such matters, as each situation is different. Your concerns about RAID-5 implementations on current motherboard chipsets are well founded. Can you get HighPoint RAID cards from dealers in NZ?

While I have not done it myself, Linux software RAID looks like a good choice for you, given that portability between machines is a goal. No added controller cost yet it doesn't lock you in to a specific motherboard/chipset.

Paging unhappy_mage, the forums resident Debian/software RAID wizard...

 

[Madwand]

I recommend planning and budgeting for a full backup. Then you could try out software RAID for yourself, in practice, and if you don't like it, get an add-on controller. The migration issue is also moot when you have a full backup.

For Windows users, the choice is pretty clear IMO -- software and on-board just don't perform as well during writes as even software-assisted HW RAID controllers such as the RocketRaid and RaidCore. If you don't care much about write performance, then of course you can use anything.

For Linux software RAID (XFS), I've had mixed results -- sometimes pretty good performance, and at other times, despite trying, I couldn't get better than around 30 MB/s writes. If you get good performance, and are comfortable with Linux, have a backup to assist further (as you should anyways) and have adequate capacity with software RAID, then why spend more?

 

[unhappy_mage]

Software raid is indeed my favorite of the three (in my price range, that is). Dedicated hardware is faster, and can provide some pretty significant benefits if your machine is overloaded, but unless you're transcoding to mpeg2 for a set-top box or something, cpu usage from raid5 shouldn't be a concern. Think of it this way - for every 4 bytes you want to write, you need to perform *one* *32-bit* xor. If your chip does 64-bit calculations, then it's 1 per 8 bytes. So at 1.6 gHz and 32 bits, you could write 6.4 GB/s to disk. Using one core. If memory could keep up with that. So even if you're doing 100 MB/s, you should only see around 6% cpu usage. Turn on 64-bit and the numbers get even less worrisome. LSR is my suggestion, but let me cover the alternatives first:

I've heard bad things about onboard RAID (particularly NVIDIA's rather inexact implementation) and don't want to be SOL if I want to upgrade the motherboard later on.

Good choice. I'd stay away from on-board driver-based RAID unless someone held a gun to my head. I just lost onboard scsi on one of my machines () and it seems to have taken the rest of the board with it.

A dedicated RAID card I'm unsure of as I've heard mixed results with it - while it would allow me to move it between machines with the least fuss (theoretically), cheap cards (ie. ones that I'll be getting as server bound components are damn expensive) supposedly shove all the work onto a CPU so in effect I've paid far too much for a software RAID setup.

Highpoint and Areca are my two favorites currently, and they cater to completely different crowds. Highpoint is driver-based, but well-done driver-based raid; it uses the host CPU to do the XORs. Areca is completely hardware-based, expensive, and fast as heck. Both are stable and work with Linux.

So. LSR. You'll need as many SATA ports available as you want to use disks, so this means adding on controller cards. The card I'm using has drivers built into the kernel, but they're fairly bad (hey, they're marked HIGHLY EXPERIMENTAL ). They appear to be "stable" in the "no data loss" sense, but STR is horrible - 50 MB/s at best off 8 disks? I get better than that off a single disk with nicer controllers... However, it appears to be a good piece of hardware, so I'm sticking with it and waiting for better driver support.

If you'd like a better-supported option, this card is only $20US, has 4 ports, and is based on the sil3114 chipset. You should be able to find a .nz dealer near you for a similar price. It's only a pci card, not pci-x or pci express, so you'd be limited to 120ish MB/s if that's a concern.

Use EVMS. You'll appreciate it later. It also supports expansion, although as always backups are suggested. I tested the expansion thru VMware (with all the disk images on a single disk ><) and it worked fine, but I haven't put my data thru the wringer yet.

Last point. What expansion slots do you have on your board of choice? Asus makes a p5something-WS that has core duo 2 and pci-x and pci-express; if I were going to buy one of these it'd be up there on my list. Also, these are really nice ethernet cards, and the company that's selling them has a bunch. They use the tg3 driver under linux, and work fine. My brother has another one arriving tomorrow

HTH, and feel free to ask questions. I did something quite like what you're planning, I think.

 

[oDii]

First up, thanks to everyone for your posts.

I think I've made my mind up - as there are no real disadvantages I'm overly concerned about, I'll just go with a software RAID setup. While I could get one of these really nice RAID cards, that's $360 that I can't spend on storage space. Especially when I can just wham the drives onto either the GIGABYTE GA-945GM-S2 or GIGABYTE GA-965P-DS3 (if I'm feeling rich), justifying $360 for just the controller seems like a waste of money. Pity, as I'd like experience with server quality components, but... price is the main factor.

Can you get HighPoint RAID cards from dealers in NZ?

Yeah, but they're almost prohibitively expensive (as mentioned above).

I recommend planning and budgeting for a full backup. Then you could try out software RAID for yourself, in practice, and if you don't like it, get an add-on controller. The migration issue is also moot when you have a full backup.

Currently at home, people already have various backup procedures - burn to DVD etc. As such, the environment is pretty forgiving so I'm not too concerned about getting it wrong a few times. If this were in a commercial environment I'd take your suggestion, but don't really see a need for it in this home environment - cheers though.

Now, disk choice - anyone have any massive likes and dislikes? I've always been keen on Western Digital because as a system builder I've built well over 100 machines, each with varying models of WD drives in them and not one has failed. While statistically that isn't much of a sample compared to what I'm sure some of you will have experience with, I can also get them pretty cheap so they're likely to go in anyway. While on the subject of Western Digital, will there be any advantage/disadvantage with a software RAID implementation and WD's RE/RE2 (RAID Edition) drives? My understanding of them is that they simply have a different firmware that limits the amount of time that they can perform error recovery processes to avoid them dropping off the array. While I'm sure that dropping an array is not good, arrays have been going along fine before the advent of these drives - are they at all necessary?

Cheers again guys.

 

[unhappy_mage]

While on the subject of Western Digital, will there be any advantage/disadvantage with a software RAID implementation and WD's RE/RE2 (RAID Edition) drives? My understanding of them is that they simply have a different firmware that limits the amount of time that they can perform error recovery processes to avoid them dropping off the array. While I'm sure that dropping an array is not good, arrays have been going along fine before the advent of these drives - are they at all necessary?

The answer is I don't know. And that makes me not want to find out. I've used Maxtor and Seagate drives with LSR, and both have worked fine. WD is a wild card in RAID arrays of any sort as far as I'm concerned and that's a wild card I don't want.

By the way, 1 U.S. dollar = 1.57 NZD. So the 2220 at $686 is $437US. Ouch.

 

[oDii]

The answer is I don't know. And that makes me not want to find out. I've used Maxtor and Seagate drives with LSR, and both have worked fine. WD is a wild card in RAID arrays of any sort as far as I'm concerned and that's a wild card I don't want.

Fair enough. I'll make the decision closer to the time I expect to buy with this in mind - looks like Seagate and Western Digital are pretty close in terms of price and reliability, so I'll go with whoever lowered their price first.