Redundant NFS storage setup (part 3) : The disappointingPERC5/E (solved?)

[Matthias Saou]

Landreth, Kevin wrote :

> Using setra won't help with writes IIRC.  Has write performance seen any
> difference?  In the applications where I use a Perc5 card, writes are my
> biggest performance problem and I'd like to see if you write performance
> is still suffering.  Using setra will help with software raid just the
> same too I believe so I wouldn't write your problem off as "solved" just
> yet.
> 
> Also, could you try ext3 with -E stride=32 (for 128K) and stride=16 (for
> 64K) and see if you still have the performance issues you had before.  I
> know this may not be ideal for your application but testing with two
> file systems provides better conclusive answers for everyone beyond a
> reasonable doubt.
> 
> Thanks for all the testing, this is really useful data you are providing
> everyone.

Writes were already much faster than reads, but in my setup I don't
care about writes, only reads... quite the opposite it seems ;-) I
haven't seen any noticeable write improvements, which makes sense, I
get 200 to 300MB/s sustained in the simple stress test I'm running,
which is more than enough for me. It's what I was seeing before.

I could try ext3, but my problem is that although RHEL5.1's release
notes mention that the size limit is now 16TB, mke2fs doesn't let me
create a filesystem on that 12TB volume. I recall reading about having
to force the creation somehow, but the man page doesn't mention
anything about the limit or how to overcome it. Tips? (I could create a
smaller partition for the tests, but it then wouldn't be a fair
comparison, not knowing how the physical disks would then be used).

Quick testing of my "real world test scenarios" has shown me that a read
ahead of 16384 (512 byte blocks, so it's 8MB) is what allows me to get
the best read speed out of the array in my setup.

I also played with the PERC's LD read ahead settings, but RA reduces
the speed noticeably, and ADRA too (although less)... so NORA
(the default) seems to be best there.

Oh, last but not least : WriteBack (WB) makes a huge difference.
Overall performance is awful if using WriteThrough (WT). But luckily WB
is the default.

I haven't seen much difference with 64k vs. 128k stripes, so I'll stick
with 128k since it seems to be the recommended value, and I will only
have files much bigger than 128kB on the filesystem anyway.

I'm glad all this helps ;-)

Matthias

-- 
Clean custom Red Hat Linux rpm packages : http://freshrpms.net/
Fedora release 8 (Werewolf) - Linux kernel 2.6.23.9-85.fc8
Load : 1.59 1.51 1.30