Need recommendation for a prebuilt 40TB server

L

lassiterb

[H]ard|DCer of the Month - June 2009
Joined
Oct 10, 2005
Messages
814
Looking for a recommendation on a 40TB server. The server will need to be able to run linux (OpenSuse 12.2 most likely). Would like hardware RAID so as not to have to deal with the pain of managing drive replacement with linux software raid. The server must also be able to handle writing a stream of data that will be roughly 1.5Gbps that will be fed to it via 10Gig ethernet.

Has anyone had any experience with any vendors that could provide something like this?

Any suggestions / comments are appreciated.
 
Gunter. said:
Approx 150MB/s sustained write.. that's pretty high for a single storage array at 40TB.

What is the budget?

I have used smaller versions of SANs from Cybernetics (relatively inexpensive compared to NetApp/LeftHand/etc)
Check http://www.cybernetics.com/Virtual-disk-storage/iSAN6012.php
& http://www.cybernetics.com/Virtual-disk-storage/iSAN6024.php
Click to expand...

Initial budget was $16000 but I am finding that to be way too low unless I was to build it myself which I am trying to avoid as I want support and a warranty on the system as a whole this time since it is for a customer and not for internal use.

Have been looking at a system from Aberdeen. A server with 30 3TB 7200RPM drives in a RAID 10 should be able to handle the sustained write speed I need and give me the needed capacity I would think. Just kills the budget coming in at a bit over $24,000. Comments?
 
Check out Aspen Systems: http://www.aspensystemsdirect.com/servers/pc/Stratum-ZSHJ2208I-ZFS-SAN-p1321.htm

They can get you 45TB of raw storage (enterprise class) with a ZFS server with 3 year 24/7 onsite parts for under $13k. We looked at them as a possible vendor before building a Napp-It all-in-one build on our new ESX server. We used their configuration as a base building guide for what we purchased. Overall they were the cheapest and best warranty we could find but they were just outside of where we could end up rolling our own build.

They are a Nexenta partner as well if you are really looking for vendor support. It will cost more obviously but that's where you end up having to weigh cost vs. support.
 
A single chassis from acmemicro like this might work for you.

I specced out 24 3TB drives (Which is 72 raw, so depending on your RAIDset) with an adaptec controller, around 12K with overkill on space.

You can often email them and ask for an upgrade not available in the dropdowns (like an areca raid card instead of adaptec)

I would suggest something a bit newer (DDR3 support in particular).
 
Are those 3TB drives SAS or SATA? Based on the initial post I would say look at SAS only drives due to the throughput needs and the dual channel nature of SAS drives.

If you're looking for a 40TB storage server why would you be looking at OpenSUSE rather than have a dedicated storage server or do an all in one build with ESX or Hyper-V? I would be VERY concerned with installing an application on a storage server which may have a negative impact on the server itself. Separation of duties in my mind is key when you are talking about 40TB.

Also, what's your backup strategy for that much data? That's something you haven't mentioned in your budget or your build.
 
I would also say nearline (7200rpm) SAS drives are highly recommended.

Use ones that are on the HCL for the RAID card / HBA you decide to use.
 
brianmat said:
Are those 3TB drives SAS or SATA? Based on the initial post I would say look at SAS only drives due to the throughput needs and the dual channel nature of SAS drives.

If you're looking for a 40TB storage server why would you be looking at OpenSUSE rather than have a dedicated storage server or do an all in one build with ESX or Hyper-V? I would be VERY concerned with installing an application on a storage server which may have a negative impact on the server itself. Separation of duties in my mind is key when you are talking about 40TB.

Also, what's your backup strategy for that much data? That's something you haven't mentioned in your budget or your build.
Click to expand...

Reason for OpenSUSE is this may be running a bit of custom software to ingest the data stream over a network socket rather than having it just use it as a file server. The handling of this data stream is its only reason for living, so no only one duty and its a big duty :) .

Also, should have been clearer in my description, the 40TB is the required space needed after any raid configuration.

Also, yes looking at nothing but NL-SAS drives for the configuration.

No backup strategy, this will be used for only short term storage of data (Aprox. 1 week).
 
Last edited:
If you're really only looking at a 1 week retention then I don't know why you don't roll your own and save a bundle. Since your budget is about $16k and this isn't long term data then just grab the same SuperMicro case they used and go to town with some NL SAS drives and a good LSI RAID card.

I'm partial to ZFS because it really can be stupid simple to use once you get the basic concepts down and you are not tied to specific RAID cards.

We found out that what we saved on building our own could have paid for a spare motherboard, a few sticks of RAM, a spare LSI card, and a few spare hard drives all sitting at the datacenter if we need it.

You can check out a pretty cheap build at www.zfsbuild.com - even if you don't use ZFS much of the hardware will still apply.
 
Gunter. said:
Approx 150MB/s sustained write.. that's pretty high for a single storage array at 40TB.

What is the budget?

I have used smaller versions of SANs from Cybernetics (relatively inexpensive compared to NetApp/LeftHand/etc)
Check http://www.cybernetics.com/Virtual-disk-storage/iSAN6012.php
& http://www.cybernetics.com/Virtual-disk-storage/iSAN6024.php
Click to expand...


Cybernetics is OK but I would look elsewhere.

We have their VTL on property and although its fairly fast and compact, the web interface is a buggy slow joke.
 
You must log in or register to reply here.
Back
Top