EMC Powerpath / RHEL 4 u6 / sd* devices high load

[Harald_Jensas at Dell.com]

Hi,

You could try updating PowerPath to EMCPower.LINUX.5.0.1.Patch.b022.tar.gz. (Available trough powerlink.emc.com)

If that does not help I would try downgrading to 4.5.1 or 4.5.3 as well.


--
Harald Jensås



> -----Original Message-----
> From: linux-poweredge-bounces at dell.com [mailto:linux-poweredge-
> bounces at dell.com] On Behalf Of Robert Jaroszuk
> Sent: 06 December 2007 10:03
> To: linux-poweredge-Lists
> Subject: EMC Powerpath / RHEL 4 u6 / sd* devices high load
> 
> Hi there.
> 
> I have a 3-node Oracle 10g RAC cluster on RedHat 4 update 6 (2.6.9-67)
> with an EMC storage connected through FC.
> The problem is, that after some time, a number of sdX devices seems to
> have a 100% load:
> 
> Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s   wkB/s
> avgrq-sz avgqu-sz   await  svctm  %util
> sda          0.02   0.02  0.66 53.87   33.93  431.09    16.97   215.54
> 8.53    59.78   22.09  18.70 101.98
> sdb          0.01   0.00  0.02  0.00    0.07    0.00     0.03   0.00
> 3.07     0.00    2.41   0.06   0.00
> sdc          0.01   0.00  0.02  0.00    0.07    0.00     0.03   0.00
> 3.07     0.00    2.64   0.10   0.00
> .
> .
> sdx          0.01   0.00  2.15  1.02    5.43    1.02     2.71   0.51
> 2.04    85.60    0.37 322.26 101.89
> sdy          0.01   0.00  0.71  0.00    4.93    0.00     2.46   0.00
> 6.94     1.02    0.74 1435.65 101.94
> 
> But server was NOT used at that time. It does nothing. It is
> impossible, that this %util was caused by real load.
> 
> I noticed that the problem lies somewhere on kernel level, because the
> number of I/Os currently in /proc/diskstats is very huge.
> 
>    8    0 sda 32652 1095 1682597 110960 2678799 774 21436528 59785187
> 4294966435 50633734 2909368627
>   65  112 sdx 106545 306 269352 30959 50429 0 50429 27440 4294967295
> 50588021 4244391545
> 
> It is the third field from the right (4294966435 and 4294967295).
> Notice that this number is 232 - 861 and 232 - 1.
> 
> I also use EMC PowerPath (EMCpower.LINUX-5.0.0-157) and I think that
> this problem may be caused
> by this software. Those devices behave correctly unless I load kernel
> modules from emc. After that it's just
> a matter of minutes when devices have 100%load in iostat.
> 
> Unfortunately, Dell and EMC support told me that they do not support
> PowerPath for RH update 6 (only for u4),
> so I can downgrade my systems (unacceptable for me) or I can pis^H^H^H
> go away...
> 
> Maybe you have any idea what can I do with that?
> 
> 
> --
> ... Robert Jaroszuk ...
> GCS/IT/O d? s: a- C++ ULB++++$ P+ L++++$ E- W++ K- N+ DI+ V-
> w M- PS+ PE Y(+) PGP-(+++) t-- 5? X R !tv b++>++++ D- y+ G++
> . http://zim.iq.pl/ . RJ735-RIPE . http://zim.iq.pl/photo/ .
> .. The superior warrior wins without fighting -- Sun Tzu. ..
> 
> 	-> New photos: http://zim.iq.pl/photo/
> 
> 
> _______________________________________________
> Linux-PowerEdge mailing list
> Linux-PowerEdge at dell.com
> http://lists.us.dell.com/mailman/listinfo/linux-poweredge
> Please read the FAQ at http://lists.us.dell.com/faq