Tuning a RAID60 array for best performance.

Harald_Jensas at Dell.com Harald_Jensas at Dell.com
Wed Aug 27 18:02:51 CDT 2008


> -----Original Message-----
> From: linux-poweredge-bounces at dell.com [mailto:linux-poweredge-
> bounces at dell.com] On Behalf Of Nicolas Chipaux
> Sent: 27 August 2008 16:03
> To: linux-poweredge-Lists
> Subject: Tuning a RAID60 array for best performance.
> 
> Hello,
> 
> My setup : PERC6/E adapter with one MD1120 enclosure on each connector full of
> 24 physical drives.
> We manage this server with MegaCLI SAS RAID Management Tool  Ver 2.00.11 March
> 10, 2008 and OMSA 5.4.0. For best performance, we switch Linux scheduler to
> cfq.
> 
> We try to get the best performance with a correct security on the RAID volume.
> That's why we choose RAID60. We are using orion (Oracle Database Load Simul.)
> to benchmark random read/write in different data size.
> 
> On the RAID60 device under Debian Linux, we got :
> 
> # ./o -run advanced -testname t -num_disks 1 -type rand -matrix detailed \
>         -size_small 8 -size_large 2048 -verbose
> Small IO size: 8 KB
> Large IO size: 2048 KB
> IO Types: Small Random IOs, Large Random IOs
> 
> Maximum Large MBPS=203.30 @ Small=0 and Large=2
> Maximum Small IOPS=821 @ Small=5 and Large=0
> Minimum Small Latency=5.75 @ Small=1 and Large=0
> 
> 
> If we setup 2 RAID6 devices (still using Debian), we got :
> 
> # ./o -testname 2vol -run normal -num_disks 2
> Small IO size: 8 KB
> Large IO size: 1024 KB
> IO Types: Small Random IOs, Large Random IOs
> 
> Maximum Large MBPS=189.81 @ Small=0 and Large=4
> Maximum Small IOPS=1241 @ Small=10 and Large=0
> Minimum Small Latency=7.68 @ Small=4 and Large=0
> 
> 
> The latency is higher be we obtain a 50% performance bump. As RAID60 add a
> layer of stripping between the spans, how can IOPS decrease ?
> 

If I am not mistaken your RAID 60 test has Large IO size: 2048 KB, and the 2 RAID 6 test has Large IO size: 1024. Larger IO size generally = less IOPS and smaller IO Size generally = less Throughput. Are you sure you are not seeing Double the IO size and see approximately "only" 1/3 less performance?

Also remember that the write penalty on many drives striped with parity RAID can be quite high for small random IO. If data on Stripe Element (disk) 1 and 2 is changed the controller will have to read Stripe Element 3 - 10 to calculate parity data for Stripe Element 11 and 12. Essentially you have 4 writes and 6 reads to update data in 2x Stripe Elements. The controller will do its best to use Write Cache to avoid this, by not writing back to disk before it can write a full stripe and by doing this avoiding to read data from drives to recalculate parity. But in a random IO scenario this is less likely to succeed. 

You might get a better performance result by creating 3x 7 drive RAID 5 arrays, spanning these to a RAID 50 and configuring the 3 remaining drives as hot-spare drives. 


> Then, we wanted to test using different stripe size in the RAID6 span of the
> RAID60 array. 

The OMSA CLI will let you specify this.

Complete Syntax

omconfig storage controller action=createvdisk controller=id raid=<c| r0| r1 | r5| r6 | r10| r50 | r60> size=<number | max | min> pdisk= <PDISKID> [stripesize=< 2kb| 4kb| 8kb| 16kb| 32kb| 64kb| 128kb>] [cachepolicy=<d | c>] [readpolicy=<ra | nra | ara | rc| nrc>] [writepolicy=<wb| wt| wc| nwc | fwb>] [name=<string>] [spanlength=<n>]


Something like:
omconfig storage controller action=createvdisk controller=id raid=r60 size=<number> \
pdisk=0:0,0:1 <insert rest of drives> \
stripesize=< 2kb| 4kb| 8kb| 16kb| 32kb| 64kb| 128kb> spanlength=12

Documentation is available here: http://support.euro.dell.com/support/edocs/software/svradmin/5.4/en/omss_ug/html/omss_ug_html.zip



Regards
Harald Jensås




More information about the Linux-PowerEdge mailing list