Cheap ways to add a JBO device to an existing server

M

mephistopoa

Weaksauce
Joined
Dec 11, 2010
Messages
122
Hi Guys,

I'm planning to expand the storage of a Dell R515 server so I'm evaluating some solutions on how to cheaply achieve this. My idea is to buy one of those JBO cases (supermicro perhaps) and connect it to the existing R515 server. In which ways can I achieve this?

I don't want to have a raid controller running on the other box as my plan involves ZFS, so how could I connect the JBO box to the server? Is there a dummy version of a JBO box just with a HBA card or something that can be hooked using a SAS cable to another box? Can I use one HBA card on each end to achieve this?

I need some suggestions how to achieve this as if I go for NFS/iSCSI I'll need to have a full PC built on this second box, and for now I'm not considering it.

Thanks!
 
What is your budget? Do you already have an HBA on the main server? Search for forum for SE3016, there is an interesting thread about this.
 
My budget is as cheap as possible as this is my lab and I'm funding it.

I've got a SAS HBA already on this server, I was testing on a DAS MD3200 a few months ago for a client so I kept the card. Are there enclosures with simply power supply and SAS connections that concentrate into one of those SF connectors so I can connect to another server?
 
it is quite cheap in the US, but shipping to the UK is a rape!

Will try to find some cheap alternatives here
 
anyone in the uk that can give me tips of where to find a similar priced enclosure?
 
mephistopoa said:
anyone in the uk that can give me tips of where to find a similar priced enclosure?
Click to expand...
Do you need 12, 16, 24 bay ?
... enclosure could be cheap in price but expander / control power board / external SFF 8088 / redundant PSU (the price of a 24 bay ...) "could cost some money" !
 
Last edited:
ST3F said:
Do you need 12, 16, 24 bay ?
... enclosure could be cheap in price but expander / control power board / external SFF 8088 / redundant PSU (the price of a 24 bay ...) "could cost some money" !
Click to expand...

I just need a cheap 12bay to begin. I just want to start testing ZFS properly and this would be my proof concept, so just no need for redundant equipment. I just need to power several SATA disks on a rack, and use a SAS cable to the HBA card on the R515 and be able to see the disks as native SAS/SATA devices so I can enjoy fully ZFS.
 
mephistopoa said:
I just need a cheap 12bay to begin. I just want to start testing ZFS properly and this would be my proof concept, so just no need for redundant equipment. I just need to power several SATA disks on a rack, and use a SAS cable to the HBA card on the R515 and be able to see the disks as native SAS/SATA devices so I can enjoy fully ZFS.
Click to expand...
in UK, a 8 bay already costs £ 895.00 ex VAT
=> http://www.xcase.co.uk/8-drive-jbod-p/jbod-8.htm

To make a JBOD, a case must include :
1x EC-CB-RM-215-6G-SAS 2U 8 x 6G SAS hotswap
1x ACC-CB-SASEXP-CK22803 6G SAS Expander CK22803 16 Port
1x ACC-CB-SASEXP-UEK-BRACKET UEK Bracket for CK12801 + CK13601
2x ACC-3W-CBL-SFF8087SB-0.6M 0.6mtr Multilan SFF8087 to SFF8087 6G
1x PSU-ZP-2u- 500 Watt Redundant
Click to expand...

Have you check on ebay.co.uk for HP MSA60 (12bay JBOD) ?
... it's a good product but it costs more than SE3016 + shipment.
cf : http://www.ebay.co.uk/itm/HP-MSA60-...0?pt=UK_Computing_Servers&hash=item5894ecf044

I made a quote for a 12bay JBOD with 650w redundant, 2 years warranty, in Luxembourg : 1 073 € / £ 825 ex VAT, without shipment... so the price is quite equal as MSA60 & SE3016, but with warranty.

Cheers.

St3F
 
ST3F said:
in UK, a 8 bay already costs £ 895.00 ex VAT
=> http://www.xcase.co.uk/8-drive-jbod-p/jbod-8.htm

To make a JBOD, a case must include :


Have you check on ebay.co.uk for HP MSA60 (12bay JBOD) ?
... it's a good product but it costs more than SE3016 + shipment.
cf : http://www.ebay.co.uk/itm/HP-MSA60-...0?pt=UK_Computing_Servers&hash=item5894ecf044

I made a quote for a 12bay JBOD with 650w redundant, 2 years warranty, in Luxembourg : 1 073 € / £ 825 ex VAT, without shipment... so the price is quite equal as MSA60 & SE3016, but with warranty.

Cheers.

St3F
Click to expand...

Thanks for the tip mate, but still a bit high comparing for example with a R515, I can get a 6 core opteron, with 12 bays + 2x 2.5 internal bays for around £800 and then it is a proper server. This time I'm envying the guys in the US, £100-200 for a 12 bay rack with all the hardware, what a cheap deal!

I think I'll need to go for the R515 then.

I was just wondering if there is a way to connect 2 servers for example using SAS HBAs? I can't think on a way for it to work but perhaps someone has already thought about this like having one of the servers running solaris, doing ZFS and then serving as a SAS target for another server. Is this possible?
 
mephistopoa said:
Thanks for the tip mate, but still a bit high comparing for example with a R515, I can get a 6 core opteron, with 12 bays + 2x 2.5 internal bays for around £800 and then it is a proper server. This time I'm envying the guys in the US, £100-200 for a 12 bay rack with all the hardware, what a cheap deal!
Click to expand...
+1

mephistopoa said:
I think I'll need to go for the R515 then.
Click to expand...
So with Motherboard, CPUs, Ram .... include ?!?

mephistopoa said:
I was just wondering if there is a way to connect 2 servers for example using SAS HBAs? I can't think on a way for it to work but perhaps someone has already thought about this like having one of the servers running solaris, doing ZFS and then serving as a SAS target for another server. Is this possible?
Click to expand...
2 servers with their own HBA, to have 1 pool = clustering

You can buy a JBOD or build one ... ; to build one you'll need :
  • case
  • expander
  • control power board
  • external SFF 8088
  • standard or redundant PSU

++
 
^^^ even with the shiping and import taxes, there is no simpler/cheaper way howto get a sas jbod

sad truth is any expander or even decent case will exceed that price in no time
and thats not counting psu or cables
 
Last edited:
ST3F said:
+1


So with Motherboard, CPUs, Ram .... include ?!?


2 servers with their own HBA, to have 1 pool = clustering

You can buy a JBOD or build one ... ; to build one you'll need :
  • case
  • expander
  • control power board
  • external SFF 8088
  • standard or redundant PSU

++
Click to expand...

1x Opteron 4234, 6C, 3.1GHz
2GB ram (will get 4x8GB for £200 anywhere else, cheaper than Dell) has 8 dimm slots
H200 controller (LSI rebranded, flash with IT firmware)
chipset AMD SR5670/SP5100

with this server I can have 12 3.5 inches hooked to the H200 and 2x 2.5 inches hooked to SATA inside the case, I'm using this set up at the moment but with a H700 and LSI cachecade with a Intel 520. Still testing though, not much to say as I didn't really stress the server.

I can get the server for around £800 or less because I buy stuff regularly from dell so I get some discounting.

Regarding connecting the servers via a SAS cable, I'm not intending to have a single pool (at least now) so my idea is to have a very fast connection to host virtual machines for example. I can have one box dedicated just for the VMs and other just for shared storage. Perhaps having a 3rd box doing replication of the second box then in the future enabling high compression and dedupe?

Is possible to connect 2 boxes via a SAS interface? My ideia first is to avoid iSCSI because of the latency and bottleneck becuase of 1gbit cards, and offload on the network card, link aggregation, way too much stuff for a simple set up like mine.

regarding the list of items to buy fopr a JBOD build, can you give me examples of the parts needed? I never stepped into this territory so I'm a bit lost where and what exactly to look for. Thanks for your help mate, much appreciated!
 
asgards said:
^^^ even with the shiping and import taxes, there is simpler/cheaper way howto get a sas jbod

sad truth is any expander or even decent case will exceed that price in no time
and thats not counting psu or cables
Click to expand...

Can you suggest something for example please?
 
mephistopoa said:
Regarding connecting the servers via a SAS cable, I'm not intending to have a single pool (at least now) so my idea is to have a very fast connection to host virtual machines for example. I can have one box dedicated just for the VMs and other just for shared storage. Perhaps having a 3rd box doing replication of the second box then in the future enabling high compression and dedupe?
Click to expand...
So why a JBOD and not a DAS ?

In Europe ...

DAS :
Norco DS-12D External 2U 12 Bay Hot-Swap 6G SAS / SATA III Rackmount
=> http://cybershop.ri-vier.nl/ds12d-external-2u-12-bay-hotswap-6g-sas-sata-iii-rackmount-p-193.html (Starting at: €435.00 Excl VAT)
Norco DS-24D External 4U 24 Bay Hot-Swap 6G SAS / SATA III Rackmount
=> http://cybershop.ri-vier.nl/external-4u-24-bay-hotswap-6g-sas-sata-iii-rackmount-ds24d-p-136.html (€645.00 Excl VAT)

//

JBOD :
Entropia External 2U 12bay 6G SAS/SATA Expander (LSI based) Rackmount RAID / JBOD Enclosure, redundant PSU 650w
=> http://4u9ur.free.fr/images/Raid5/2u_12/Entropia_2u_12.jpg (€1150.00 Excl VAT)
Norco : DS-24ER External 4U 24 Bay 6G SAS/SATA Expander Rackmount
=> http://cybershop.ri-vier.nl/external-4u-24-bay-6g-sassata-expander-rackmount-ds24er-p-216.html (€1,645.00 Excl VAT)

Is possible to connect 2 boxes via a SAS interface?
Click to expand...
I've never ever seen that.

Cheers.

St3F
 
ST3F said:
So why a JBOD and not a DAS ?

In Europe ...

DAS :
Norco DS-12D External 2U 12 Bay Hot-Swap 6G SAS / SATA III Rackmount
=> http://cybershop.ri-vier.nl/ds12d-external-2u-12-bay-hotswap-6g-sas-sata-iii-rackmount-p-193.html (Starting at: €435.00 Excl VAT)
Norco DS-24D External 4U 24 Bay Hot-Swap 6G SAS / SATA III Rackmount
=> http://cybershop.ri-vier.nl/external-4u-24-bay-hotswap-6g-sas-sata-iii-rackmount-ds24d-p-136.html (€645.00 Excl VAT)

//

JBOD :
Entropia External 2U 12bay 6G SAS/SATA Expander (LSI based) Rackmount RAID / JBOD Enclosure, redundant PSU 650w
=> http://4u9ur.free.fr/images/Raid5/2u_12/Entropia_2u_12.jpg (€1150.00 Excl VAT)
Norco : DS-24ER External 4U 24 Bay 6G SAS/SATA Expander Rackmount
=> http://cybershop.ri-vier.nl/external-4u-24-bay-6g-sassata-expander-rackmount-ds24er-p-216.html (€1,645.00 Excl VAT)


I've never ever seen that.

Cheers.

St3F
Click to expand...

Alright, you nailed it :)

What kind of "controller" those boxes have? The first one for example has 3 connections, can I hook it to 3 different hosts? What is the point of 3 connections? Some sort of link aggregation?
 
mephistopoa said:
What kind of "controller" those boxes have? The first one for example has 3 connections, can I hook it to 3 different hosts? What is the point of 3 connections? Some sort of link aggregation?
Click to expand...
No controller.

There are 3 backplanes. ; each can handle 4 drives
These backplanes are each connected to a SAS external port SFF 8088 ... so there are 3 connectors ! ;)
To handle 3 external connector, you must own, as you want / can :
- Dual External SAS Port (LSI SAS 9205-8e) + one more Dual Exteran SAS or One External SAS Port & one Internal ( LSI SAS 9212-4i4e HBA )
- Triple or more External SAS Port ( LSI SAS 9201-16e HBA )

In SAS, the big difference between DAS and JBOD is that you can't Daisy Chained DAS.
=> it's Direct Attached ...

However, if you go with DAS, with 2 dual external SAS cards or one Quadro external card, you'll be able to use 3 ports to connected to DAS and the last external port to a JBOD when the time will have come...

Cheers.

St3F
 
ST3F said:
No controller.

There are 3 backplanes. ; each can handle 4 drives
These backplanes are each connected to a SAS external port SFF 8088 ... so there are 3 connectors ! ;)
To handle 3 external connector, you must own, as you want / can :
- Dual External SAS Port (LSI SAS 9205-8e) + one more Dual Exteran SAS or One External SAS Port & one Internal ( LSI SAS 9212-4i4e HBA )
- Triple or more External SAS Port ( LSI SAS 9201-16e HBA )

In SAS, the big difference between DAS and JBOD is that you can't Daisy Chained DAS.
=> it's Direct Attached ...

However, if you go with DAS, with 2 dual external SAS cards or one Quadro external card, you'll be able to use 3 ports to connected to DAS and the last external port to a JBOD when the time will have come...

Cheers.

St3F
Click to expand...

Yeap, I got it. It is because I'm so used to those ready made solutions from Dell, like the MD-3200 that is a DAS with a raid controller, I've several clients using it at the moment but the thing is damn expensive and during the floods in thailand I could not source Dell certified drives to use on it. They block any non dell firmware drive so I've got in a very crappy situation with some clients. At least their normal raid controllers Like the H200 and H700 don't have this crap blocking any more. Well going back, I mentioned the MD3200 because it gets all data over a single SAS 8088 cable so I think the only way to do the same in this situation would be having a RAID controller on the BOX, which again increase a lot the price. The 3 port LSI card is damn expensive, the dual port card I buy from dell normally costs £70-80 so it is a much better solution. I just don't like much the idea of having 3 different cables connected to a single box if they don't add any redundancy.

How this box has only 1 SAS connection?

http://www.ebay.com/itm/SGI-RACKABL..._Disk_Arrays&hash=item1c299f640a#ht_988wt_906

It has some sort of controller on it right? That is the thing I was looking for to be honest, all SAS/SATA devices being router through a single SAS cable. In this case then again the JBOD looks better, but the price is so salty! I still don't get why they are so expensive. 1150 euros buy me a R515 fully equipped. The only problem in my opinion here is iSCSI, I like the idea but I haven't got much experience with it so as ZFS in the long term for me is attractive becuase of ARC/L2ARC and ZIL on SSD, the box would be destined for a very demanding environment where IOPS is a must.

I've seen some tests showing latency being quite rubbish on iSCSI even on 10GB ethernet.

by the way, thanks again for all your explanations, I owe you a pint!
 
As an eBay Associate, HardForum may earn from qualifying purchases.
mephistopoa said:
YHow this box has only 1 SAS connection?

http://www.ebay.com/itm/SGI-RACKABL..._Disk_Arrays&hash=item1c299f640a#ht_988wt_906

It has some sort of controller on it right?
Click to expand...
The right name is not "Controller" but as I told you from the beginning : "Expander" ;)

114164.jpg
468406-B21.jpg
LD0000907989.jpg


dev24fig1.gif
Chenbro-CK13601-36-port-SAS-Expander.jpg


mephistopoa said:
I've seen some tests showing latency being quite rubbish on iSCSI even on 10GB ethernet.
Click to expand...
Depends to the CPU too ...

++
 
As an eBay Associate, HardForum may earn from qualifying purchases.
Don't know much about the R515.....

But it looks like there is a 12bay version available yes?

Now if you can get a good price on them as you say....

Do the backplanes on them have sff-8087 connections?

If so, maybe you could

Get 2 of these
http://www.ebay.com.au/itm/320812305199?ssPageName=STRK:MEWAX:IT&_trksid=p3984.m1438.l2649#ht_2210wt_1038

And 2 of these
http://www.dell.com/downloads/global/products/pvaul/en/dell-6gbps-sas-ita-spec-sheet.pdf

Which I believe you already have one?

And 4 x sff-8088 to sff-8088 cables

Join the two servers together that way...

Bonus is the second R515 can just be another ESXi box booting off say a USB stick.

.
 
Hmmm just looked at the manual for the R515 / R510
http://server.ru/html/products/dell/servers/rack/dell-poweredge-r515/guide/install.htm

Seems the backplane uses a combination of both 2 x sff-8087 AND 2 x sff-8484 ....

Bit of a bugger... But still workable I guess.

You will need to do the maths on HOW cheap you can get another R515 off Dell... And then what the associated parts / cables / connectors etc can be had for in your country to do the above conversion.

.
 
I contact one of the guy who sells SE3016
... he says shipment = 100 $ / pcs

=> SE3016 + shipment + VAT + fees = $384 (320 €) to France.

Cheers.

St3F
 
Keep in mind you don't always have to 'buy it now'. Mine was advertised at $250 and I bid back and forth and he went down quite a bit.
 
If the chassis only has one SAS minisas cable SFF8088, then it probably has a SAS expander built in.
 
Stanza33 said:
Don't know much about the R515.....

But it looks like there is a 12bay version available yes?

Now if you can get a good price on them as you say....

Do the backplanes on them have sff-8087 connections?

If so, maybe you could

Get 2 of these
http://www.ebay.com.au/itm/320812305199?ssPageName=STRK:MEWAX:IT&_trksid=p3984.m1438.l2649#ht_2210wt_1038

And 2 of these
http://www.dell.com/downloads/global/products/pvaul/en/dell-6gbps-sas-ita-spec-sheet.pdf

Which I believe you already have one?

And 4 x sff-8088 to sff-8088 cables

Join the two servers together that way...

Bonus is the second R515 can just be another ESXi box booting off say a USB stick.

.
Click to expand...

It sounds good, I'll need to confirm which connector my R151 uses Monday.
 
Quick question about the SE3016. The 2 SFF-8088 are for cascading or can I actually manage to connect 2 different servers to the same expander and each have access to certain number of disks?
 
I apologize for my noobness but what do you do with that SGI unit? How does it connect?

If Im thinking correctly JBOd just means the pc will see a bunch of drives? Or can you raid these?
 
I have no idea - that's just what the diagrams say - in/out. For marley1, it's an expander, so when you plug a HBA into the SAS/in connector, it sees a bunch of drives. If the HBA wants to raid them it can, but you can jbod if you want.
 
Yes,its definitely a JBOD configured enclosure -at the third photo is clearly visible how backplane cables are connected to an "SAS-JBOD module"

btw.
the price is bargain, at leas from my POV :)
 
You must log in or register to reply here.
Back
Top