![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Hey guys, I've been lurking on the forums for the past month or so while I've been researching ZFS NAS builds for home use and I was hoping I could get some feedback and advice before pulling the trigger and ordering stuff. I've done a lot of reading but I am getting to the point where I have reached the limits of my own personal experience and knowledge.
The main goal for this project is to come up with a scalable and fault tolerant solution that can be used to consolidate data that I have spread across many different drives on a Win7 desktop PC which also runs VMWare Server for 3 small virtual machines. I do not plan on using encryption, deduplication or compression but I can see the possibility of using snapshots eventually. This NAS will be multipurpose so I already know that I will be struggling to balance cost, capacity and performance. My budget top end is $2500 USD for a minimal initial capacity of 8TB usable.
Currently the bulk of my data is in a 4TB CIFS share that I use for storing media and embarrassingly enough, it consists of 3 USB drives passed through to a virtual machine, concatenated into a single logical volume and then shared out via Samba. I also have around 700GB of data on 2 other unprotected SATA drives that I would like to move to the NAS as well. I would classify 300GB of this data as unstructured that can be moved to a CIFS share without a problem. The other 400GB is application and game installations that would be moved to an iSCSI lun or split into 2 separate iSCSI luns. If I can get at least 8TB of usable storage then that should give me enough to consolidate and then plenty of room to grow.
What I am proposing is an ESXi All-in-One box with a Solaris 11 Express virtual machine running as a NAS appliance. The NAS virtual machine would share out an NFS datastore for ESX, a few CIFS/NFS shares for media and other unstructured data and an iSCSI lun to the Win7 desktop to replace the multiple aging SATA drives.
So far my base build looks like this and comes in just under $2200 USD.
From everything that I have been reading this is a pretty standard configuration and should work well for what I want it to do. The drive configuration however is where I am having a difficult time making a decision and keep going back and forth between different options.
Right now I am leaning heavily towards option 2 since it exceeds my minimum capacity requirements and it fits in very nicely with the SASUC8I controller and the NORCO RPC-4220 chassis (8 drives fills the SASUC8I and 2 backplanes of the chassis). As far as performance is concerned, it really comes down to whether or not I should expect iSCSI to perform well enough to replace the local SATA drives on the Win7 desktop. If this is not a realistic expectation then performance becomes less of a concern since anything is going to give me better performance and fault tolerance than what is currently in place.
I ran a couple of quick CDM benchmarks against the 2 SATA drives that I would like to see replaced with iSCSI luns. From what I can tell, the performance on these drives is already pretty poor compared to some of the iSCSI benchmarks that I have seen in other threads, which is what gave me the idea in the first place. It would be nice to protect the drives with ZFS and have the ability for extending them as they fill up instead of having to replace them with larger drives.
I apologize for the length of the post but I wanted to give as much info about what I am attempting to accomplish.Any recommendations or thoughts on fine tuning this build are greatly appreciated.
The main goal for this project is to come up with a scalable and fault tolerant solution that can be used to consolidate data that I have spread across many different drives on a Win7 desktop PC which also runs VMWare Server for 3 small virtual machines. I do not plan on using encryption, deduplication or compression but I can see the possibility of using snapshots eventually. This NAS will be multipurpose so I already know that I will be struggling to balance cost, capacity and performance. My budget top end is $2500 USD for a minimal initial capacity of 8TB usable.
Currently the bulk of my data is in a 4TB CIFS share that I use for storing media and embarrassingly enough, it consists of 3 USB drives passed through to a virtual machine, concatenated into a single logical volume and then shared out via Samba. I also have around 700GB of data on 2 other unprotected SATA drives that I would like to move to the NAS as well. I would classify 300GB of this data as unstructured that can be moved to a CIFS share without a problem. The other 400GB is application and game installations that would be moved to an iSCSI lun or split into 2 separate iSCSI luns. If I can get at least 8TB of usable storage then that should give me enough to consolidate and then plenty of room to grow.
What I am proposing is an ESXi All-in-One box with a Solaris 11 Express virtual machine running as a NAS appliance. The NAS virtual machine would share out an NFS datastore for ESX, a few CIFS/NFS shares for media and other unstructured data and an iSCSI lun to the Win7 desktop to replace the multiple aging SATA drives.
So far my base build looks like this and comes in just under $2200 USD.
- SUPERMICRO MBD-X8SIL-F-O
- Intel Xeon X3430
- 16G Unbuffered DDR3 1333 Memory (4x4)
- Intel SASUC8I PCI-Express x8 SATA / SAS Controller Card
- OCZ Agility 2 50GB SATA II MLC SSD (L2ARC for faster reads)
- NORCO RPC-4220 4U Rackmount Server Chassis
- CORSAIR Builder Series CMPSU-500CX 500W Power Supply
- 8 x Samsung F4 HD204UI 2TB drives, or Hitachi 5K3000 2TB drives for ZFS
- NETGEAR ProSafe GS108T-200NAS (LACP, Jumbo Frames)
- Reusing existing 40G WD Raptors for ESX Installation.
From everything that I have been reading this is a pretty standard configuration and should work well for what I want it to do. The drive configuration however is where I am having a difficult time making a decision and keep going back and forth between different options.
- 1 x 10GB raidz1 vdev (5 x 2TB drives) 8TB usable
- 2 x 8GB raidz1 vdev (4 x 2TB drives) 12TB usable
- 4 x 4GB raidz1 vdev (4 x 1TB drives) 12TB usable
Right now I am leaning heavily towards option 2 since it exceeds my minimum capacity requirements and it fits in very nicely with the SASUC8I controller and the NORCO RPC-4220 chassis (8 drives fills the SASUC8I and 2 backplanes of the chassis). As far as performance is concerned, it really comes down to whether or not I should expect iSCSI to perform well enough to replace the local SATA drives on the Win7 desktop. If this is not a realistic expectation then performance becomes less of a concern since anything is going to give me better performance and fault tolerance than what is currently in place.
I ran a couple of quick CDM benchmarks against the 2 SATA drives that I would like to see replaced with iSCSI luns. From what I can tell, the performance on these drives is already pretty poor compared to some of the iSCSI benchmarks that I have seen in other threads, which is what gave me the idea in the first place. It would be nice to protect the drives with ZFS and have the ability for extending them as they fill up instead of having to replace them with larger drives.
I apologize for the length of the post but I wanted to give as much info about what I am attempting to accomplish.Any recommendations or thoughts on fine tuning this build are greatly appreciated.
Code:
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 31.562 MB/s
Sequential Write : 53.298 MB/s
Random Read 512KB : 20.614 MB/s
Random Write 512KB : 29.459 MB/s
Random Read 4KB (QD=1) : 0.389 MB/s [ 94.9 IOPS]
Random Write 4KB (QD=1) : 1.385 MB/s [ 338.2 IOPS]
Random Read 4KB (QD=32) : 0.737 MB/s [ 179.9 IOPS]
Random Write 4KB (QD=32) : 1.395 MB/s [ 340.6 IOPS]
Test : 1000 MB [D: 78.8% (220.1/279.5 GB)] (x1)
Date : 2011/03/16 22:11:35
OS : Windows 7 [6.1 Build 7600] (x64)
Code:
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 46.579 MB/s
Sequential Write : 45.719 MB/s
Random Read 512KB : 27.722 MB/s
Random Write 512KB : 31.348 MB/s
Random Read 4KB (QD=1) : 0.515 MB/s [ 125.7 IOPS]
Random Write 4KB (QD=1) : 1.028 MB/s [ 250.9 IOPS]
Random Read 4KB (QD=32) : 1.111 MB/s [ 271.1 IOPS]
Random Write 4KB (QD=32) : 0.998 MB/s [ 243.6 IOPS]
Test : 1000 MB [F: 92.0% (274.0/298.0 GB)] (x1)
Date : 2011/03/16 22:16:23
OS : Windows 7 [6.1 Build 7600] (x64)
Using the builtin iscsi initiator in win7 allows you to make the network storage look like a local disk.