Want to build a file storage server (raid question)

I'm looking to build a server that will do nothing more than store files and i want to keep it as cheap as possible, but make it reliable.

so, i want to use raid, but i'm not sure which i should go with.

raid 1
pros: would be easy (most mobo's with raid support this outta-the-box). would have reliability.
cons: paying for extra hard drive, but not getting extra storage.

raid 5
pros: would have more storage for my money over raid 1, and would be reliable
cons: have to buy 3 drives minimum. cards/mobo's with raid 5 support are expensive.

raid 0 + 1
pros: supposedly fastest and most reliable of all raid config options. (i don't care about fast, just reliable and a lot of storage)
cons: have to buy 4 drives minimum, 2 of which offer no extra storage, only reliability.


anyway, what do you guys recommend? i'd like to have a lot of storage, while having some reliability in case something would happen. (it's not critical data, just a pain if i'd have to get it again.... i'm a digital packrat). and i'd also like to keep it fairly cheap... or at least get my money's worth.

thanks.

RAID5

you pretty much covered the drawbacks
but once youve made that investment the reliability isnt much different
just cant afford to loose 2 drives at the same time

I would point out the increasing efficiency as you add drives
3 x 100GB = 200GB
6 x 100GB = 500GB
12 x 100GB =1100GB
thus it make more sense the more space you need

of course you cant dynamically add drives to an array for any reasonable amount of money
(the intial RAID Controller investment)
so youd need to back them up, then add new drive and rebuild the array if you chose not to fill it up from the start
having a "hot spare" is over hyped unless its an enterprise environment
for a single user or even a small LAN Id use all the channels and have an extra drive handy to physically replace any that go down

infact I do

Ice Czar said:

of course you cant dynamically add drives to an array for any reasonable amount of money
(the intial RAID Controller investment)
infact I do

Click to expand...

Most the new raid 5 capabale controllers will allow you to dynamicaaly add drives to the raid 5 array. They just need to be the same size, or larger than the ones already in teh array.

i've searched for some raid 5 pci cards and they all seem to be pretty expensive. anyone know of any that are good for a decent price?

any other input on the other options?

thanks for the replies.

How much is "alot of storage"?

Are you going to use Windows XP for the "server" OS?

I would recommond that you go with a hardware raid controller, but don't forget that software raid is an option.

I would prefer as much as i can get... maybe 300 gig or so.

i will use linux as the server OS.

If you want hardware raid 5, you're going to spend money. No questions about it.

Regularly you can find 3ware 8 and 12 port hardware raid cards on the auction site for around 200-300 dollars. (the non cache variety) but they support raid 5. you have to consider if you're paying over a $100 for the drives, and you want say 6 of them, $300 for a controller isn't that much.

i have a 3ware 7800-8 that i bought for $50. i filled it with 80 gig drives, and had a 600 gig array.

QM

I was just thinking about this my self. I used to think software raid was out of the question. There as a post on slashdot about this sort of thing and there were a number of people saying how mature and efficient linux software raid has become. I have also come to believe that you can get good performance out of a software raid setup after reading some examples - and yes, it will use your CPU for the parity calculations, but if it is a dedicated storage server that will not be a problem. Here is the kernel output of a linux 2.4.9 system, hardware is a duron 700:

Note the raid-5 parity calculation rate of 2.6GB/second. Imagine what a more modern processor can do. The mdadm tool for linux raid supports hot spares and can email you if any thing should happen.
The only thing it doesn't't have is a stable online expansion function yet. However if you want something to be a big file system you can use LVM (logical volume manager) for linux which allows you to stitch, move around and resize physical volumes. After some reading, it allows you to say expand your array by joining two raid 5 arrays together. Combined with some 4 port SATA cards from ebay you can have your self a very cheap solution considering how expensive a hardware solution is...
You can also boot of a linux software raid 0 or 1 array. You can however boot off a software raid5 array if you put the bootloader and kernel on a compact flash drive combined with a compact flash to ide converter - also available on ebay cheaply.

Another attractive fact: You can increase the performance of the array by upgrading your motherboard and CPU..

retrofitter said:

I have also come to believe that you can get good performance out of a software raid setup after reading some examples - and yes, it will use your CPU for the parity calculations, but if it is a dedicated storage server that will not be a problem.

Click to expand...

Agree ... even software RAID 5 under Windows isn't that bad if its a dedicated storage server ... what else is it going to use those CPU cycles for?

retrofitter said:

You can however boot off a software raid5 array if you put the bootloader and kernel on a compact flash drive combined with a compact flash to ide converter - also available on ebay cheaply.

Click to expand...

Hmmm ... I like it ...

I'm using Linux software RAID 5 on four 300 gig Diamondmax 10 drives. When writing, the CPU hit for parity calcs is about 15% on an Athlon XP 1900, using LVM2 on top of the RAID. Just to put it into perspective, the max real world write speed to the array that I've seen is about 33 MB/sec. Reads saturate the 33MHz PCI bus.

retrofitter said:

Note the raid-5 parity calculation rate of 2.6GB/second. Imagine what a more modern processor can do. The mdadm tool for linux raid supports hot spares and can email you if any thing should happen.

Click to expand...

It is true that modern CPUs are rather quick at parity calculations, mostly due to the face that the embedded procs that do parity on the board are running at ~100 MHz.

Anyway, it is not the CPU that is th bottleneck for RAID5, it's a bus issue. If you do SW raid, much more data has to traverse the PCI bus than with a HW card.

However, I 'just' got a Promise SX4-M and I am NOT happy with the speed. i am running 4 SP1614C's in RAID5 and I get write speeds off ~50MB/s which is LESS than my single 80 Maxtor drive. I think that I might have some configuration issue, which I have yet to figure out.

[edit]:
just found a link to a nice projects over at SR:
http://variant.ch/phpwiki/WikiBlog/2005-02-27

He gets ~80MB/s on a A64-3200+ using RAID5 on a 2TB array

You're right about bus saturation being a bottleneck, but I actually think this is generally a win for software raid. The NForce 4 builtin SATA controller doesn't go through the PCI bus and is VERY fast. The link to the article from SR shows he's probably bottlenecking on the SI 3114 controller on the PCI bus. I haven't seen a lot of good PCI-E hardware raid controllers, but I expect to see some simple SATA controllers show up for PCI-E that will also obviate the problem.

If you look at the case of a similar system to the one posted with a broadcom PCI based hardware raid controller, it will have less performance because it goes through the PCI bus. The built in SATA ports aren't bottlenecked. This will change when lower cost PCI-E hardware raid controller become available, but you always seem to get more options for bypassing I/O bottlenecks with software RAID.

Thanks,
Mike

since the issue was raised
thought Id toss this in since I saw it the other day
really only applicable if youve already invested considerable funds into PCI-X cards and want to employ PCI-E

(thus a very limited market)

PCIE-PCIX-7S
Seven-slot PCI Express to PCI-X Expansion System
"PCIE-PCIX-7S is a 7-slot PCI Express to PCI-X expansion system that provides system developers an easy-to-use migration path to high speed PCI express capability while maintaining slot capacity for PCI-X and PCI cards. The system features a standard height 4x PCI Express host card, a 7-slot 5U 19-inch rackmount chassis and 5-meter cables. The expansion system is ideal for PCI-X/PCI-based multimedia and test applications that will benefit from the higher performance of PCI express."

I see highpoint is about to release a 4 port SATA controller for PCI-E, that will be a great way to add drives to a software RAID system under Linux. The NF4 boards have 4 SATA II ports that bypass the PCI bus for great performance. But most them of them resort to something like the Silicon Image 3112 for 4 additional ports, and that means the 32 bit 33 Mhz PCI bus can become a bottleneck.

This new card can remove that bottleneck and should make for an incredibly fast software RAID implementation under Linux.

Thanks,
Mike