EMC storage array tweaking

[Peter Grandi]

>>> On Wed, 05 Mar 2008 13:08:53 -0600, Jeff Macfarland
>>> <jmacfarland at nexatech.com> said:

[ ... ]

jmacfarland> You've got 1 GB of write cache on a cx300.

That does not help much for sequential IO. Conversely the ondisk
cache is very important for special reasons (mainly that it is
not used as a cache, see below).

jmacfarland> Having cache on or off of the physical disks
jmacfarland> themselves wont make anything faster

You would be surprised :-). Enabling the ondisk cache helps a
lot for sequential IO both because it is ondisk (latency) and
how it is used by the drive firmware.

Anyhow, to see how much it matters, try sequential writes after
disabling the cache with 'hdparm -W0' on a simple ATA/SATA disk
directly attached to a Linux PC (note how the problem is still
present but smaller with 'O_DIRECT').

Problem is that especially with ATA/SATA back-to-back reads (or
writes) have a good chance of happening only if there is a
''track'' buffer on the drive (the latency between drive and
page cache manager is often too high), and the on-disk write
cache is used as a r/w track buffer too (and in some cases the
use of the on disk cache as a read cache or a track buffer is
disabled too when one disables the write cache).

jmacfarland> except perhaps if you're filling the cache and a
jmacfarland> forced flush is popping up.

For bulk sequential IO the CX300 cache does not matter because
what you say is then what happens all the time: the cache is
always full after a short while (it takes a few seconds to fill
1GB either reading or writing) and all the IO thereafter is just
cache spills (triggered by the upper/lower thresholds you
mention below).

A cache may then devolve into a bulk buffer helping smooth out
sequential transfer rate variability, but at least the
applications I care about generate or request data faster than
the disk can deal with.

jmacfarland> Check your cache settings on the array to make sure
jmacfarland> write caching is on and you have the upper and
jmacfarland> lower flush limits correctly.

I'll have a look anyhow (the CX300 has been setup apparently by
EMC consultants, so anything could have happened, like a setup
involving RAID3/4 :->).

Anyhow, I am still interested in any suggestion on how to check
and if necessary tweak the ondisk cache of the disks on a
Dell/EMC CX300...