Megaraid with brakes on?

Dan Pritts danno at internet2.edu
Wed Jan 3 16:19:44 CST 2007


> We have a Dell  poweredge 2850 server, running as webserver and we're
> not satisfied with the speed of our RAID hardware. We're not sure if
> we're expecting too much from it or if there's something which is not
> working good.
> 
> We're using RedHat 4 ES, with the ext3 FS. Our RAID controller is Dell
> PowerEdge Expandable RAID controller 4 (rev 06) (according to lspci)
> and we have setup a RAID level 5 with 3 SCSI disks, Seagate Barracuda
> 10k 73gb
> 
> We're using the machine for serving a few million 3k files, sizing up
> ~30gb via HTTP, using the lighttpd webserver.


It's probably not really the raid controller that is falling down here;
it's your choice of RAID5 and your many-small-files workload.

I presume that the file access is not sequential within directories, but
random - if it's sequential, then much of what i say doesn't directly apply.

with millions of small files, your disks are spending lots of time seeking.
This is probably the primary cause of your performance difficulty.

I'm not sure how good the ext3 file system is with loads like that;
you might also see improvement with another filesystem.  This is worthy
of  investigation.  Also, depending on how random your workload really
is you may see improvement by tuning the kernel's metadata buffering
(inode & directory caches; i'm not sure exactly what is available).

You also appear to be doing a non-trivial number of writes.  small
writes (log files?) to a RAID5 device really slow down things; the RAID
controller needs to first read the existing stripe, then write the entire
new stripe after merging the newly written data and recalculating the
RAID5 parity.

Here's how I would approach this:

Assuming that you have lots of simultaneous requests, you would probably
see significant improvement by adding a fourth 73G disk, converting the
system to use a pair of hardware RAID1 devices, and doing a software
concatenation of the two RAID1's (or, just segmenting your data across
two separate filesystems).

alternately, you could do a RAID10 in hardware; but generally adding
RAID0 is useful to increase raw throughput on large files.  You don't
need that, and my guess is that it would be slower than the concatenated
RAID1's for your workload.

As long as you're taking downtime to do this, you might try adding a
third pair of disks in another RAID1.   i'm not sure they'll fit in
that chassis, and if it did it would leave you without the ability to
have a hot-spare.  


Dan Pritts, System Administrator
Internet2
office: +1-734-352-4953 | mobile: +1-734-834-7224



More information about the Linux-PowerEdge mailing list