Advice on storing ~3TB of stuff.

E

^eMpTy^

2[H]4U
Joined
Jul 21, 2004
Messages
3,233
I have around 3TB of multimedia scattered across 8 drives on 2 different machines. Last night one of my 500GB drives ate itself and I'm once again thinking about RAID/backup solutions to prevent this from happening again.

I've been reading forums and window shopping at newegg for a couple of days now and the right thing to do has yet to jump out and reveal itself.

So my question is, given the following situation, and a budget of less than $1000, what would you do?

The Situation:
- 2 computers to work with (my desktop and my media center), both can hold 6 drives each and have plenty of SATA ports.
- Already have 6 500GB drives, 1 750GB drive, 1 640GB drive. All are full or close to full.
- One of the 500GB drives just died.
- I am extremely technically competent, but have no hands-on experience with RAID.

I feel like RAID 5 is just going to lead to more problems. However, RAID 1/mirroring on disk seems like an enormous waste of space (maybe this is just flawed logic on my part?). So I'm really not sure what to do. It would be nice to have something that "just works".

What would you do?
 
RAID is NOT a backup.

Now that we have that out of the way, I am quite partial to hardware R5 arrays. I have had very good experiences with the RR23XX series cards. While they are not true hardware raid (CPU does the calculations) they are fast enough for me(RR2320 on an old A64 3000+ saturates gigabit).

If you want true hardware R5 at a good price, look at the Dell PERC 5/i. A lot of people here are partial to WHS with no hardware form of RAID, personally I am a bit leery of having a Windows box hosting and all systems having full read/write access to my data.

Thus, my setup is as follows:
-Linux running on my fileservers (Protects from Windows viruses that NOD32 may not catch, plus I never actually do anything on my fileservers other than IRC through SSH and the web frontend for my usenet downloader)
-All but one upload folder is read only to SMB clients. This prevents someone hooking into my network with one of those nasty viruses that destroys media files. This also means that I manage files using the command line.


But, that is just me. I'm running two arrays currently. One is 2.03TB (8x320) and the other is 6.36TB (8x1TB). I'll be upgrading the 320GB drives to 1.5TB soon, yielding 9.77TB of usable space.
 
The efficiency of RAID 5 is really appealing. But now I'm thinking that for the money it would cost me to get a solid RAID card, I could just buy more drives and keep an identical set of drives on my two machines and sync them every night.

Anybody have any experience just syncing machines over a network?
 
Agreed with Dew, RAID is not a backup solution and should never be used as such.
There are a few scenarios where RAID can help you keep a live system going (through fault tolerance and redundancy) but there are a ton of failure modes which it cannot protect you from.

My personal backup favourite for a non live/ non business system is to use standard 3.5" SATA drives equivalent/larger in size to the data to be backed up.
These are kept offline when not in use to minimise wear.
Its such a cheap system that you can use 2 sets of backup drives, alternating between them for even better long term resiliancy.
If you need day to day backups as well, use a live drive to backup to and still use the offline storage.

I am not a fan of other backup mediums at home for 3 reasons.
1) not really that reliable / more prone to failure (software and hardware), more modes of failure
2) cost
3) unnecessary complexity / difficulty
 
Nenu said:
Agreed with Dew, RAID is not a backup solution and should never be used as such.
There are a few scenarios where RAID can help you keep a live system going (through fault tolerance and redundancy) but there are a ton of failure modes which it cannot protect you from.

My personal backup favourite for a non live/ non business system is to use standard 3.5" SATA drives equivalent/larger in size to the data to be backed up.
These are kept offline when not in use to minimise wear.
Its such a cheap system that you can use 2 sets of backup drives, alternating between them for even better long term resiliancy.
If you need day to day backups as well, use a live drive to backup to and still use the offline storage.

I not a fan of other backup mediums at home for 3 reasons.
1) not really that reliable / more prone to failure (software and hardware), more modes of failure
2) cost
3) unnecessary complexity / difficulty
Click to expand...

So you guys view RAID 5 as more of a performance/availability solution and not a way to protect data?
 
^eMpTy^ said:
So you guys view RAID 5 as more of a performance/availability solution and not a way to protect data?
Click to expand...

It will only protect against the mechanical failure of a single hard drive. Take that to mean what you want.
 
Agreed.
Also any unwanted changes made to your data are not backup up by any RAID solution.
With sometimes hard work, you may be able to recover changes made (ie corruption, accidental or malicious deletion etc.) but if you cannot, you will need to have a decent backup to recover from.
 
I have a WHS and I don't need to access any of my data outside of my home (mostly just for streaming blu-rays/dvds/music between my HTPC, gaming rig and laptop. Are you guys saying that it's best to leave it not connected to the internet in this case?

If so, how would i do this? Stupid question, I know, but if it's plugged into a switch that receives internet, how do i make it so that it's not online but all of the other computers are without disabling the port making it not accessible from all of the other computers?
 
I dont follow where you are going with this.
If you want a decent backup system, dont mistake RAID for one.
 
Nenu said:
I dont follow where you are going with this.
If you want a decent backup system, dont mistake RAID for one.
Click to expand...

Nenu, I agree with your statement, but my understanding of the OP question, is how to avoid losing data if a HDD fails. As a mattter of fact he never used the word "backup".

In this case RAID or WHS duplication will do the trick right?

So for a $1000 budget I would buy a mobo with integrated GPU, and 6 SATA ports, 2 G of mem, a cheap dual core CPU, decent Case\PSU a couple of 1.5TB HDD and Windows home server.

Throw 4 of your 500GB HDD in the case and you got plenty of redundancy. You just tell WHS which folders you want duplicated and WHS does the the work for you.

On top of that, later on if you want backup or off site storage you can add a 1TB external HDD.


Total Price: $1,022.93

You can probably find stuff cheaper or maybe even re-use more existing parts.

The cool thing is that with this you will add:
- 24\7 File server to your network
- Low power file services (~-70-80Watts)
- Remote access capability if required
- Selectable redundant file storage (no need to worry about disk setup)
- Ability to run unattended backups of all your home computers
- Ability to run backup and store them off site (like for your digital pictures)

I built something very similar using mostly existing parts, and I can't tell you how happy I am with it,

Best of luck
 
Dew said:
RAID is NOT a backup.

personally I am a bit leery of having a Windows box hosting and all systems having full read/write access to my data.



you can configure whs to have any level of access you want on different machines. its not just a big nas box if you dont want it to be. whs can also make system backups for you if you choose. if one drive fails you dont lose the whole "array", and you also dont lose half or more of your space for redundancy like you do with raid. it also doesnt cost an armand a leg to build a whs box. most people fail to realize most raid cards are not true raid cards. neither are any mb based raid solutions. software raid is useless and hardware raid is too damn expensive. so whs provides the solution.
Click to expand...
 
Trepidati0n said:
1TB drives are $100
WHS is $100
Click to expand...

Trep, no need to scream. Did you not notice me saying that there were prob cheaper deals out there?

Also if 1 TB is $100, I would still go with the slightly more expensive 1.5TB.

Oh yeah, could you please provide the links to the $100 WHS?

SpeedyVV said:
You can probably find stuff cheaper or maybe even re-use more existing parts.
Click to expand...
 
SpeedyVV said:
Oh yeah, could you please provide the links to the $100 WHS?
Click to expand...

Basically every store carries them at that price now, but start at Zipzoomfly.


Anyways, I second the WHS recomendation.
 
Sorry for barking...but your whole price list was pretty out of whack. I understand you being in Canada...but a massive majority of readers are not. I just get sorta edgy lately...need to back away from the keyboard.

BTW...makes sure you look at the hot deals section at least once a day. All your questions in regards to prices are usually answered.

Right now my recommendation to most people building a file server is to do EXACTLY what I did for my box. What I did is the most cheap and simple solution out there.

1. Get a mobo with six SATA slots onboard (make sure you can run the extra 2 out of non raid). Quite a few of the MATX gigabyte boards do this.

2. Spend very little on the processor. The WHS box, if used as intended needs very little CPU time. In my case...I have downclocked my processor to 1.2 GHz.

3. RAM...you don't need much. 1GB is fine and it can be a single dimm. You don't need dual channel performance.

4. Get a quality single RAIL PSU. This will allow you to put more HDD's on the supply without worrying about rail saturation during startup. You can get a nice 500W corsair (which will support 16 drives, assuming 2 amp startup limit). This will cost you less than $0.1/Watt. Not a bad price.

5. The case is subjective. Me personally, I like rack cases with hotswap bays. Once you have one as a file server you sorta realize this is what you should do. My alternate is the Antec 300 with extra fans. Will knock a $100 off the deal.

6. Use WHS. It takes little time to get up and running and pretty much ZERO time maintaining. It almost gets to the point of scary when you realize you went three+ weeks without even looking at it.

7. Get the biggest HDD's you can. $/GB of space is less important when thinking about need extra "ports". Just a little while ago 1.5TB drives were a bit more expensive than 1TB per GB. However, when you factored in that it was a 3 drives + extra port +power versus 2 drives, the answer wasn't as clear.
 
Another vote for WHS. I have a WHS box with 8TB total, works very well.

For an off site back up solution of absolutely critical information I also use Jungle Disk for WHS.
 
Trep, apologies accepted and forget about it.

The main thing that I want to OP not to loose focus on is the fact that he\she can get a nice WHS setup within his budget.

What you are claiming is that it is possible to get it well below $1000 in the US. Even Better ;-)
 
You must log in or register to reply here.
Back
Top