MD3000 configurations? (performance issue) FIXED

[justin]

For the sake of future reference, I'd like to note i fixed the problem.

The MD3000 now performs like a champ, the problems were all down to an 
issue with one
of its two controllers, which refused to read faster than 3mb/sec. 
Oddly, this problem didn't
show up on any diagnostics! One would think that the controller could 
realize it was not performing
when read speeds drop so much, and take itself out of the loop. Oh well.
LUNs re-assigned manually to the other controller magically worked to 
spec. Until I replace
the controller or the cable I won't know for sure what the culprit was.

According to iozone, it now can read RAID10 or RAID0 over a single HBA 
card at about 160
and write at 250 mb/sec. Unlike linux software raid, performance on 
unrelated LUNs does not
suffer when one is being hammered. I'm really happy to see that. I was a 
big user of internal
hard drives setup with software raid, but hated the way the machine 
would get really slow for
the lightest load when doing a giant I/O operation on a single /dev/md 
device.
Performance with parallel tests also indicates that two hosts and two
cards, or two hosts and four cards are probably going to get suck close 
to the total throughput
advertised by Dell out of this unit (1000+ mb/sec and perhaps thats full 
duplex),
but I'm still running benchmarks to be sure.

What isn't clear to me yet is quite how seamless the RDAC or multipath 
tools are going to be
in the event of an i/o path failure. Frankly, I don't trust RDAC it 
seems really hairy and badly
undocumented. I'll probably try to test suse 10 x64 which apparently 
works well on this box
without any Dell code, using multipath tools instead.

final note:

I've seen comments online (SmugMug blog for instance) that one should 
disable the READ cache
on external enclosures because you want to maximise any memory they have 
for write caching
(being battery backed up and all that) the host has far more memory for 
read cache anyway.
This isn't documented very well but it is possible to disable read cache 
and/or read-ahead on a
per LUN basis, by using the SMCli command line utility, but not the Java 
management interface
supplied by Dell.

u1_fd at dslr.net wrote:
> Under kernel 2.6.18, as provided by RHEL5, CentOS5 or Fedora Core 5, I 
> can't make
> a new MD3000 perform when paired to a Dell 1950 with the provided SAS 5 HBA
> card (PCI-X) and using X535 quad core cpus, etc etc.
>
> 1. Driver release numbers are all over the place.
> The MD3000 resource CD invents and provides driver release 4.x for the 
> LSI 1068
> based controllers, but kernel versions come with 3.04.02, and Dell 
> itself offers source
> to "3.02.83.12-5" for rhel5/suse.
>
> None of the source from Dell (4.x or 3.02.83.12-5) can be compiled 
> against any kernel
> past 2.9.18 because of the change to the workqueue API.
>
> 2. The worst problem is when the array does come up, read performance is 
> awful.
> hdparm -t reports 3mb/sec read, and iozone confirms it.
> Running vmstat during iozone shows blocks out during write of ~100k but 
> during
> reads blocks in are just 3000 per second.
>
> 3. Write is mid 1xx mb/sec, which is slower than the integrated PERC 5 
> on the same
> machine with a simple RAID1 pair of the same 146gb sas drives.
> worse, write speed does not improve with more disks. It is no faster 
> with two
> disks in a RAID10 LUN, as it is with all 16 disks!
>
> I'm thinking under linux these things are lemons.
> If you currently use them, can you help & reply with
>
> uname -a
> modinfo mptsas | grep version
> hdparm -t on a block device that comes from a LUN
> lspci | grep 1068
>
> any other comments on speed, failover, etc, that you've observed?
>
> I've seen other topics here that suggest the PCI-X vs PCI Express change 
> may have killed things,
> but even if I get read performance to mid 100s MB/sec its still very 
> upsetting considering a single
> chip PERC 5 that comes with every dell is doing better and scales, to 
> some extent, with the spindles.
>
> Last question, we've an unused MD1000 , does _that_ offer good linux 
> performance?
>
> thanks!!
>
> _______________________________________________
> Linux-PowerEdge mailing list
> Linux-PowerEdge at dell.com
> http://lists.us.dell.com/mailman/listinfo/linux-poweredge
> Please read the FAQ at http://lists.us.dell.com/faq
>
>