Possible to do a RAID 0 of Flash cards?

R

rabident

Limp Gawd
Joined
Oct 29, 2004
Messages
300
Kingston has 4GB 133x Compact Flash cards. They sell CF -> IDE adaptors pretty cheap and old 4 channel IDE RAID 1/0 controllers aren't much either.

I was thinking of getting 4 cards (16GB) and building a RAID 0 set out of them. Should be dead quite, low heat, have great seek time, and because they're all working in parallel STR should be good as well.

I was eagerly awaiting Gigabyte's i-RAM 2, but it doesn't look like it's coming out.

I want my games to load faster and my PC to be quieter.
 
Hmm, I'm thinking RAID increases speed do the the physical limitations of a mechanical drive. I don't think it would make a difference with flash.... I could be wrong though....
 
Yes, it'll work. However:
The highest speed I've seen out of a flash drive is 20MB/s. (133x * 150KB/s = 19...MB/s, which is up there.) A perfect RAID0 of four of them will get you 80MB/s, which is doable with a much cheaper, NORMAL hard drive as long as you don't bog it down with crap, keep it defragged, and maybe even shell out the extra for a 36GB Raptor. A real-world setup won't be 100% efficient anyway.

On the other hand, seek times with your RAID array will be measured in microseconds at worst, not milliseconds. You're talking orders of magnitudes faster than even a 15K SCSI drive. :eek:

Edit, and FYI:
I'm running one of those adapters, with 2x Sandisk 2GB UltraII CF cards. This is first-hand experience. :)
 
mavalpha said:
Yes, it'll work. However:
The highest speed I've seen out of a flash drive is 20MB/s. (133x * 150KB/s = 19...MB/s, which is up there.) A perfect RAID0 of four of them will get you 80MB/s, which is doable with a much cheaper, NORMAL hard drive as long as you don't bog it down with crap, keep it defragged, and maybe even shell out the extra for a 36GB Raptor. A real-world setup won't be 100% efficient anyway.

On the other hand, seek times with your RAID array will be measured in microseconds at worst, not milliseconds. You're talking orders of magnitudes faster than even a 15K SCSI drive. :eek:

Edit, and FYI:
I'm running one of those adapters, with 2x Sandisk 2GB UltraII CF cards. This is first-hand experience. :)
Click to expand...

Interesting, this has certainly peaked my curiosity.
 
I have this idea too!!!

I'm thinking of using an Adapte 1200A which has 2 IDE channels and then getting 2 x CF-IDE converter which have master and slave configuration.

I'm wondering whether using the dual CF converter with master and slave will reduce the transfer rate. comments??
 
I know this could be done easily under linux with software raid and usb sticks - no additional hardware needed. Perhaps there is similar software for windows, I don't know. I only have 1 usb flash drive at the moment though, but I might buy more just to play around with this.

I have run HD Tune on a couple flash devices when testing for ReadyBoost. My OCZ Rally 4GB USB stick has a seek time of ~5.5 milliseconds, and a Patriot 2GB SD card through a USB cardreader has a seek time of ~1.2 milliseconds. [ms figures from memory, might not be exactly right] From what I've read, the Rally is designed for high transfer rates and not latency. If there is interest, I can rerun the tests and post transfer rates.

I think that a really successful flash-raid setup would need to have carefully selected flash parts to minimize latency and maximize throughput. Problem is, it is very hard to find the stats without buying the cards. I haven't seen any databases of info, except for a couple reviews and the ReadyBoost Compatibility Chart which only has a few entries with benchmark figures. I imagine there will be more structured benchmarks, ala storagereview for flash as it becomes more prominent as a primary storage medium. In the meantime, we could start posting the specs on any flash devices we have.
 
Nice idea. My corsair turboflash gives

34,6MB/s average read
random access 0.6ms

So 4 of these together would equal one very high performing system.
 
As an eBay Associate, HardForum may earn from qualifying purchases.
I was looking at this Windows Software RAID Guide and trying to replicate it under Vista. In the Disk Management snap-in, I can't even remove or create partitions on the removable devices. I can only format. Anyone know how if this functionality just doesn't exist in vista? (I have ultimate, btw)

edit: I gathered up all the flash I could find and ran HD tune on them. Surprise on the results with the OCZ rally, I'm not sure why it is so much better than before, but I re-ran it 4 times to make sure it was consistent.

Patriot 1GB SD card - 8.2MB/s 2.3ms
Patriot 4GB SD card - 7.8MB/s 1.4ms
OCZ Rally 4GB - 19.5MB/s 0.6ms!

I also ran hdtune on the 4GB SD and Rally simultaneously and they maintained the same performance.
 
using CF card as a hard disk is different from using USB as Ready Boost...
 
naro said:
using CF card as a hard disk is different from using USB as Ready Boost...
Click to expand...

no shit. I am talking about using flash of any kind as a storage device, not readyboost (though I have tried that for a little fun). I am very curious to see which is faster, flash via ide or flash via usb.
 
okie.. USB is gonna lose out cos its theoretical max is 480Mbps = 60MB/s

Meanwhile the IDE interface has a max bandwidth of 133MB/s. And Compact Flash card can be used on an IDE slot and the max bandwidth is its own speed. The fastest CF card right now is the Sandisk Extreme IV which was in-house tested by Sandisk to have a max of 40MB/s.

I believe Windows cant' do a software Raid on USB mass storage device.
 
Also remember flash cards have limited Writes/Reads so using it as a main drive they can have a very short life...
 
will disabling pagefile help reduce the writes on the drive containing the OS?
 
naro said:
will disabling pagefile help reduce the writes on the drive containing the OS?
Click to expand...

If the system has enough memory you can disable paging to disk, it probably isn't paging too much in the first place. I guess every little bit helps :)
 
taqueso said:
If the system has enough memory you can disable paging to disk, it probably isn't paging too much in the first place. I guess every little bit helps :)
Click to expand...

You can't disable paging; that's one of the biggest and most never-ending myths there is. Windows will always have a pagefile regardless of what you say, set, do, tweak, configure, choose, decide, etc. The pagefile is just one small component of the virtual memory subsystem and as such cannot be simply disabled with a setting.

You can do some things to reduce the effect of the need to page data to the physical hard drive, but you can't stop it absolutely. Sorry... :p
 
You can RAID anything, even floppies if you want. Just do it in software.

I take that back...I don't think you can RAID tape drives together.....
 
naro said:
okie.. USB is gonna lose out cos its theoretical max is 480Mbps = 60MB/s

Meanwhile the IDE interface has a max bandwidth of 133MB/s. And Compact Flash card can be used on an IDE slot and the max bandwidth is its own speed. The fastest CF card right now is the Sandisk Extreme IV which was in-house tested by Sandisk to have a max of 40MB/s.

I believe Windows cant' do a software Raid on USB mass storage device.
Click to expand...

But you would be using more than one USB port, so if you RAID 4 USB flash drives you would have a theoretical max of 60MB/s x 4 = 240MB/s.. And either flash device will be limited to the speed of the flash, not the interface.. So I don't see the problem. However it probably will have to be a non windows Raid.
 
Lazn_Work said:
But you would be using more than one USB port, so if you RAID 4 USB flash drives you would have a theoretical max of 60MB/s x 4 = 240MB/s.. And either flash device will be limited to the speed of the flash, not the interface.. So I don't see the problem. However it probably will have to be a non windows Raid.
Click to expand...

480Mbps is the bus speed, not the per device speed. Each flash drive would need it's own USB 2.0 bus if you wanted 480mbps for each one.
 
naro said:
I'm wondering whether using the Dual CF card adapter will affect the throughput cos IDE is divided in to Master and Slave but the bandwidth is still the same..
Click to expand...
Not much. It's 133MB/s aggregate for the entire channel, which is still 66MB/s per device.
taqueso said:
I was looking at this Windows Software RAID Guide and trying to replicate it under Vista. In the Disk Management snap-in, I can't even remove or create partitions on the removable devices. I can only format. Anyone know how if this functionality just doesn't exist in vista? (I have ultimate, btw)
Click to expand...
It depends how you have it accessed. XP also didn't allow partitioning of USB devices or removable media. Using a converter should make it a 100% functional HD though, right down to partitioning.
edit: I gathered up all the flash I could find and ran HD tune on them. Surprise on the results with the OCZ rally, I'm not sure why it is so much better than before, but I re-ran it 4 times to make sure it was consistent.

Patriot 1GB SD card - 8.2MB/s 2.3ms
Patriot 4GB SD card - 7.8MB/s 1.4ms
OCZ Rally 4GB - 19.5MB/s 0.6ms!

I also ran hdtune on the 4GB SD and Rally simultaneously and they maintained the same performance.
Click to expand...
That's because the Rally is a so-called dual-channel flash drive. If you open it up, it's got two chips- and they're accessed simultaneously.

http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=290079887007
I recommend female plugs, to make cables easier to manage. The male you linked earlier plugs straight into the motherboard, making a very tight squeeze for most installations. Also, I bought mine for a carputer. This is a high-vibration environment, so I don't want ANYTHING plugged in directly.
 
As an eBay Associate, HardForum may earn from qualifying purchases.
naro said:
so in fact using a Dual CF to IDE adapter wont affect the performance?
Click to expand...

It might of each CF card was able to push over 60MB/sec(120MB/sec aggregate), but seeing as your aggregate throughput per channel is lower than 50MB/sec with a theoretical maximum of 133MB/sec, you'll be fine.
 
I was considering using 2-ports per controller for flash drives. So 2 usb ports on the motherboard, plus 3 of these supporting 8 drives.

I was thinking about going with 2G or 4G Rally drives because they have decent price/GB and decent speed, but the one I have is failing the verify read about 1/4 of the way into the drive when I run iozone on it. So either the one particular drive I have is shit, all the Rally drives are shit, or all usb flash drives are shit. I guess whether this would work out would depend on which one it is.
 
My only issue with USB drives is the high CPU usage. Using CF to IDE your CPU usage would be very low.
 
May I check what will you all be using if you're doing a software Raid for your USB devices??

Cos i dun think you can run OS off it
 
I am going to be using linux md (multiple device) driver. It should boot fine from a USB RAID set.
 
I was suggesting using a cheap highpount rocketraid. You can install windows on it by integrating the textmode drivers or loading from a floppy, or you could just use it as a second drive and install games on it. Personally, I won't be doing it because I am happy with the loading times on my 160GB Dell Raptor. $300 for 16GB is a bit expensive for my tastes. But if you are wanting to do a media center PC for an entertainment center, the entire system on a single 8GB or a R0 2x 4GB setup makes perfect sense. Since it would need almost no local storage.
 
Dew said:
I was suggesting using a cheap highpount rocketraid. You can install windows on it by integrating the textmode drivers or loading from a floppy, or you could just use it as a second drive and install games on it. Personally, I won't be doing it because I am happy with the loading times on my 160GB Dell Raptor. $300 for 16GB is a bit expensive for my tastes. But if you are wanting to do a media center PC for an entertainment center, the entire system on a single 8GB or a R0 2x 4GB setup makes perfect sense. Since it would need almost no local storage.
Click to expand...

Media center PC linux distro's can fit on a 256MB flash, and get loaded to ram on boot. No point in raiding anything.

In reality, raiding flash drives together is nothing more than a fun project. This isn't something I would ever use to actually install an OS on or run games from.
 
Dew said:
I was suggesting using a cheap highpount rocketraid. You can install windows on it by integrating the textmode drivers or loading from a floppy, or you could just use it as a second drive and install games on it. Personally, I won't be doing it because I am happy with the loading times on my 160GB Dell Raptor. $300 for 16GB is a bit expensive for my tastes. But if you are wanting to do a media center PC for an entertainment center, the entire system on a single 8GB or a R0 2x 4GB setup makes perfect sense. Since it would need almost no local storage.
Click to expand...

i'm thinking of doing that... but i hop to snatch a good deal on ebay instead.

anyway, was wondering if it i use CF card for OS, should i install softwares on the CF cards or on another drive?
 
Interesting discussion. It occurred to me that maybe RAID 5 with the ability to rebuild the array might be better than RAID 0, since the likelihood of one of these drives failing is higher than a standard hard disk.
 
I was interested in this about 6 months ago, or so. Problem I ran into was the speed limit...not the speed of the IDE bus or the CF card, but the CF to IDE converter. I tried two brands, and after phone discussions with the companies making them, they revealed that the converter has a 3MB/sec speed limit.
I'll try and find the two companies I was looking at.

Ok, found one of them:
http://www.addonics.com/products/flash_memory_reader/adsacf.asp
^ This is the SATA version, but it too has the 3MB/sec speed limit in converting that their IDE version has. I can't remember what the other company was, but they had the same limit.

Just to avoid confusion, I do mean MB, not Mb.

When I get home from work, I'll try to find the emails I got from them after the phone calls.
 
runner6m said:
I was interested in this about 6 months ago, or so. Problem I ran into was the speed limit...not the speed of the IDE bus or the CF card, but the CF to IDE converter. I tried two brands, and after phone discussions with the companies making them, they revealed that the converter has a 3MB/sec speed limit.
I'll try and find the two companies I was looking at.

Ok, found one of them:
http://www.addonics.com/products/flash_memory_reader/adsacf.asp
^ This is the SATA version, but it too has the 3MB/sec speed limit in converting that their IDE version has. I can't remember what the other company was, but they had the same limit.

Just to avoid confusion, I do mean MB, not Mb.

When I get home from work, I'll try to find the emails I got from them after the phone calls.
Click to expand...

damn.. my dreams shattered..
 
Uh oh.. and I just purchased a couple CF-IDE converters off ebay. Maybe USB is the way to go afterall. I will, of course, post results when I get them. I was under the impression that the CF cards had an IDE controller build in, this making the CF-IDE converter a pretty simple affair. This is actually the reason I was leaning towards CF, I thought that there would be better latency than with USB.
 
I got my CF-IDE adapters in the mail today. They are from this guy on eBay, although there is no metal shield like in the picture. After reading about the possible 3MB/s limit, I decided to wait and check things out before buying a bunch of flash cards. I only have 1 CF card, a 512MB sandisk of unknown speed rating. I think it is from 2003, because it says "SANDISK 03" in the corner in small letters.

First, I testing the CF card in my SanDisk ImageMate CF USB reader. It should be the same age as the card, since they were purchased at the same time. This was very dissapointing: 2.1MB/s read and 12.1ms access time. Ouch.

Next, I put the card in my LianLi 9 or 16 or whatever-in-one internal USB card reader. This time I got an average read speed of 7.2 MB/s, access time of 11.9ms. This looks more like it, although the access time is pretty much shit for the purpose of this project. But at least I know the card is capable of transferring >3MB/s.

Now I finally broke out one of the CF-IDE converters. I set it up as master, and got it all hooked up. I don't mean that I can't get Vista to start at all; it starts loading, but eventually the access light on the adapter turns on solid, and the progress bar spins forever. I tried several of the adapters, and fiddled with the bios some, but Vista just won't boot with the card in (and it won't detect it if I try to "hot" insert it, which you aren't supposed to do anyway). In safe mode, it loads a bunch of drivers (last one is crcdisk.sys. but internet searches say that it is always last in safe mode) and hangs. It boots fine with an empty adapter hooked up.

Anxious to prove that I didn't just buy a bunch of worthless crap off ebay, I shut down one of my linux boxes and installed the adapter in it. It boots, and during HDD detection, which takes quite a bit longer than usual, I see the following:
Code: 
Probing IDE interface ide1...
hdc: SanDisk SDCFB-512, CFA DISK drive
ide1 at 0x170-0x177,0x376 on irq 15
hdc: max request size: 128KiB
hdc: 1000944 sectors (512 MB) w/1KiB Cache, CHS=993/16/63, DMA
 hdc:<4>hdc: dma_timer_expiry: dma status == 0x21
hdc: DMA timeout error
hdc: dma timeout error: status=0x58 { DriveReady SeekComplete DataRequest }
ide: failed opcode was: unknown
 hdc1

This was unsettling and I suspect the reason Vista couldn't handle it, but I just used hdparm to turn DMA off. Not having DMA is pretty bad, 99.9% cpu spent in hardware interrupts during the file creation portions of bonnie++ runs. I tried turning DMA back on, but I got another of the DMA timeouts. Changing from mdma2 to mdma1 or mdma0 mode did not fix anything.

Here is what hdparm reports for those interested:
Code: 
sodium t # hdparm /dev/hdc

/dev/hdc:
 multcount    =  0 (off)
 IO_support   =  1 (32-bit)
 unmaskirq    =  1 (on)
 using_dma    =  0 (off)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    = 256 (on)
 geometry     = 993/16/63, sectors = 1000944, start = 0
sodium t # hdparm -i /dev/hdc

/dev/hdc:

 Model=SanDisk SDCFB-512, FwRev=HDX 2.27, SerialNo=004123B0403A0553
 Config={ HardSect NotMFM Removeable DTR>10Mbs nonMagnetic }
 RawCHS=993/16/63, TrkSize=0, SectSize=576, ECCbytes=4
 BuffType=DualPort, BuffSize=1kB, MaxMultSect=1, MultSect=off
 CurCHS=993/16/63, CurSects=1000944, LBA=yes, LBAsects=1000944
 IORDY=no, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  mdma0 mdma1 *mdma2
 AdvancedPM=no
 Drive conforms to: Unspecified:

 * signifies the current active mode

sodium t # hdparm -tT /dev/hdc

/dev/hdc:
 Timing cached reads:   1116 MB in  2.00 seconds = 557.81 MB/sec
 Timing buffered disk reads:   20 MB in  3.20 seconds =   6.26 MB/sec

Now, the numbers: I got 8.2MB/s sequential writes (read with iostat -k 4), and 6.6MB/s reads, with pretty low cpu overhead. I have no idea why I am getting faster writes than reads, but I trust the iostat output and dd reports the same results.

I used this utility I found called Seeker to measure random access time. Bonnie++ is supposed to give random access times, but it was reporting >100mS, even when then entire disk was cached. I had the same problem with bonnie when I was trying some USB flash sticks, sometimes even getting 250+ms readings. So I don't believe it. If anyone knows a better or more widely known benchmark than this seeker thing, let me know. Seeker reports 11.5-12.0ms random access time. For comparison, this same PC with a 250GB hitachi SATA drive gets 12.96ms, and RAID-1 of two 250GB hitachi SATAs is 12.20mS. Here is an example run on the flash card:
Code: 
Seeker v2.0, 2007-01-15, http://linux.inet.hr/how_fast_is_your_disk.html
Benchmarking /dev/hdc [488MB], wait 30 seconds.............................
Results: 83 seeks/second, 11.94 ms random access time

My conclusion is that this is a crappy flash card designed for high transfer rates (in 2003 terms) at the expense of random access times. It apparently doesn't support DMA. The adapter seems to work great, and there is definitely not a 3MB/s limit.

Next step is buying two faster CF cards. I ordered a pair of Extreme IV 2GBs from some guy in Hong Kong at a ridiculously low price, and I expect to get scammed on that one and probably won't ever see them. Of course, if I do get them, I will buy more. :D Can anyone recommend a fast, reliable, and semi-cheap CF card in the 2-4GB range? I am considering this Transcend 120x 2GB card because it scores pretty well on this list of CF benchmarks and it is pretty damn cheap for the speed/size. I do have some reservations because I recently had a Transcend USB stick fail on me, though it had seen quite a bit of use. I don't think I can justify buying the Extreme IVs at regular price, it is just too much to pay for what is really just a fun project.
 
As an eBay Associate, HardForum may earn from qualifying purchases.
Dew said:
While I hate to reference toms: http://www.tomshardware.com/2005/08/18/accelerated_compact_flash/page6.html
8MBytes/sec on a 60X CF Card (60x150KBytes = ~8.8MBytes/sec maximum rated speed)
Click to expand...

That's cool, glad to see it performed better than the manufacturer told me.
My post wasn't intended to discourage anyone from trying, merely to point out another choke point in the system and hope anyone interested in it would at least check on this before ordering a pile of expensive stuff.
 
nice job taqueso....

i can't wait for ur review of the the Sandisk Extreme IV 2gb. These are great.

are you going to run Raid on them? if you receive them.
 
You must log in or register to reply here.
Back
Top