Slow PERC4 performance
Nick Stephens
nick at ceiva.com
Tue May 23 13:28:14 CDT 2006
I have also verified that we are running the latest firmware on both
controllers. Any ideas?
Is this something I should be contacting my support rep about? My
initial fear is that the limited linux support might not cover this
however at this point I cannot tell if it is a driver or a hardware
issue.
Nick
-----Original Message-----
From: linux-poweredge-bounces at dell.com
[mailto:linux-poweredge-bounces at dell.com] On Behalf Of Nick Stephens
Sent: Monday, May 22, 2006 10:39 AM
To: linux-poweredge at dell.com
Subject: Slow PERC4 performance
Hi all,
We are setting up our 3rd PE1850 + PV220s linux fileserver, but this
newest one has slightly different raid hardware and seems to be running
much slower than our previous installations. Our 2 other systems like
this are also PE1850s, but utilize PERC 4/Di and PERC 3/DC. This new
1850 uses a PERC 4/DC and a PERC 4e/Si. Also, the older boxes are
running Redhat 9, whereas I am attempting to use Fedora Core 5 on the
new one.
Everything on the system seems to go ok until it comes to the disk
access on the PV220s. Mind you, this fileserver is 2x as large as the
previous ones (1.7TB), but the amount of disk space in use is exactly
the same as the box I am testing it against, so I don't believe that
should factor in.
I noticed the problem during our peak hours when smbd is at its highest
load transferring files to our windows servers. Whereas the older boxes
would achieve a light load on them, we never ran into issues where they
would stop serving. On the new box it reaches a similar load and then
simply stops serving files, causing other servers in the chain to lock
up with requests not being met. I have tried the latest version of
samba to ensure that it wasn't a direct problem with it, but it seems
to not be limited to samba, but rather to anything that accesses that
disk. To this end it makes me think that I either have a bad raid
configuration (though it is seemingly the same as the other 2 I have
running), bad drivers, or possibly bad hardware.
Thank you so much for you help!
Here are some tests that I have run to compare:
(this is a simple find script that walks the disk looking for files not
chowned to a user, and then chowns them appropriately)
For the sake of this test, I created a directory of 24gb on the PERC
4/DC filesystems, and on the PERC 4e/Si filesystem. You will see that
it is much faster access on the 'local' filesystem vs the PV220s.
PV220s find script
----------------------
# time ./findjacked2.sh
real 4m22.550s
user 0m0.720s
sys 0m3.928s
LOCAL find script
-----------------------
# time ./findjacked2.sh
real 1m47.667s
user 0m21.733s
sys 1m29.174s
PV220s df
------------
[root at blizzard bin]# du -sh /usr/local/ceiva/kfile/NICK/large/
24G /usr/local/ceiva/kfile/NICK/large/
'LOCAL' df
-------------
[root at blizzard bin]# du -sh /usr/local/NICK/large/
24G /usr/local/NICK/large/
NEW 1850 release / kernel information
---------------------------------------
Fedora Core release 5 (Bordeaux)
Linux blizzard 2.6.16-1.2111_FC5smp #1 SMP Thu May 4 21:35:09 EDT 2006
i686 i686 i386 GNU/Linux
NEW 1850 LSMOD
---------------------
Module Size Used by
ipv6 246561 18
nfsd 204497 17
exportfs 9665 1 nfsd
autofs4 22981 1
i2c_dev 12741 0
i2c_core 24769 1 i2c_dev
nfs 198061 1
lockd 59465 3 nfsd,nfs
nfs_acl 7745 2 nfsd,nfs
sunrpc 140925 13 nfsd,nfs,lockd,nfs_acl
dm_mirror 24081 0
joydev 13441 0
dm_mod 55129 1 dm_mirror
video 18885 0
button 10577 0
battery 13253 0
ac 8901 0
lp 16265 0
parport_pc 29413 0
parport 38281 2 lp,parport_pc
floppy 61829 0
nvram 12617 0
sg 36317 0
e1000 103853 0
ehci_hcd 33229 0
uhci_hcd 33233 0
e752x_edac 13889 0
edac_mc 18069 1 e752x_edac
ext3 121417 4
jbd 56917 1 ext3
megaraid_mbox 33745 5
megaraid_mm 14829 1 megaraid_mbox
sd_mod 20417 7
scsi_mod 129001 3 sg,megaraid_mbox,sd_mod
NEW 1850 MEGARAID INFORMATION
---------------------------------------
*******Information Of Logical Drive 0*******
Logical Drive : 0( Adapter: 0 ): Status: OPTIMAL
---------------------------------------------------
SpanDepth :01 RaidLevel: 5 RdAhead : Adaptive Cache:
DirectIo
StripSz :064KB Stripes : 13 WrPolicy: WriteBack
Logical Drive 0 : SpanLevel_0 Disks
Chnl Target StartBlock Blocks Physical Target Status
---- ------ ---------- ------ ----------------------
0 00 0x00000000 0x1113e000 ONLINE
0 01 0x00000000 0x1113e000 ONLINE
0 02 0x00000000 0x1113e000 ONLINE
0 03 0x00000000 0x1113e000 ONLINE
0 04 0x00000000 0x1113e000 ONLINE
0 05 0x00000000 0x1113e000 ONLINE
0 08 0x00000000 0x1113e000 ONLINE
0 09 0x00000000 0x1113e000 ONLINE
0 10 0x00000000 0x1113e000 ONLINE
0 11 0x00000000 0x1113e000 ONLINE
0 12 0x00000000 0x1113e000 ONLINE
0 13 0x00000000 0x1113e000 ONLINE
0 14 0x00000000 0x1113e000 ONLINE
OLD 1850 MEGARAID INFORMATION
---------------------------------------
*******Information Of Logical Drive 0*******
Logical Drive : 0( Adapter: 1 ): Status: OPTIMAL
---------------------------------------------------
SpanDepth :01 RaidLevel: 5 RdAhead : Adaptive Cache:
DirectIo
StripSz :064KB Stripes : 12 WrPolicy: WriteBack
Logical Drive 0 : SpanLevel_0 Disks
Chnl Target StartBlock Blocks Physical Target Status
---- ------ ---------- ------ ----------------------
0 00 0x00000000 0x0887c000 ONLINE
0 01 0x00000000 0x0887c000 ONLINE
0 02 0x00000000 0x0887c000 ONLINE
0 03 0x00000000 0x0887c000 ONLINE
0 04 0x00000000 0x0887c000 ONLINE
0 08 0x00000000 0x0887c000 ONLINE
0 09 0x00000000 0x0887c000 ONLINE
0 10 0x00000000 0x0887c000 ONLINE
0 11 0x00000000 0x0887c000 ONLINE
0 12 0x00000000 0x0887c000 ONLINE
0 13 0x00000000 0x0887c000 ONLINE
0 14 0x00000000 0x0887c000 ONLINE
NEW 1850 EXT3 INFORMATION
--------------------------------
Filesystem volume name: /
Last mounted on: <not available>
Filesystem UUID: 7e47c2ba-e1b4-4823-a2ca-356512b4cc2f
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index
filetype needs_recovery sparse_super large_file
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 17088192
Block count: 17075079
Reserved block count: 853753
Free blocks: 9231830
Free inodes: 16594724
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 1024
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 32736
Inode blocks per group: 1023
Filesystem created: Wed May 17 07:08:50 2006
Last mount time: Thu May 18 17:21:57 2006
Last write time: Thu May 18 17:21:57 2006
Mount count: 5
Maximum mount count: -1
Last checked: Wed May 17 07:08:50 2006
Check interval: 0 (<none>)
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
First orphan inode: 1578710
Default directory hash: tea
Directory Hash Seed: 387ddc56-0acc-49b1-9085-242344739098
Journal backup: inode blocks
OLD 1850 EXT3 INFORMATION
--------------------------------
Filesystem volume name: iFile
Last mounted on: <not available>
Filesystem UUID: 7add9ee6-3965-48bb-a8a7-22286cbdfa79
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal filetype needs_recovery
sparse_super
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 196804608
Block count: 196780177
Reserved block count: 0
Free blocks: 18851111
Free inodes: 171640778
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 32768
Inode blocks per group: 1024
Filesystem created: Fri Feb 4 16:22:31 2005
Last mount time: Wed Apr 12 20:32:07 2006
Last write time: Wed Apr 12 20:32:07 2006
Mount count: 8
Maximum mount count: 25
Last checked: Tue Feb 8 18:27:29 2005
Check interval: 15552000 (6 months)
Next check after: Sun Aug 7 19:27:29 2005
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal UUID: <none>
Journal inode: 8
Journal device: 0x0000
First orphan inode: 0
NEW 1850 DMESG RAID INFO
--------------------------------------
SCSI subsystem initialized
megaraid cmm: 2.20.2.6 (Release Date: Mon Mar 7 00:01:03 EST 2005)
megaraid: 2.20.4.7 (Release Date: Mon Nov 14 12:27:22 EST 2005)
megaraid: probe new device 0x1000:0x1960:0x1028:0x0518: bus 2:slot
12:func 0
ACPI: PCI Interrupt 0000:02:0c.0[A] -> GSI 39 (level, low) -> IRQ 19
megaraid: fw version:[351X] bios version:[1.10]
scsi0 : LSI Logic MegaRAID driver
scsi[0]: scanning scsi channel 0 [Phy 0] for non-raid devices
Vendor: DELL Model: PV22XS Rev: E.17
Type: Processor ANSI SCSI revision: 03
scsi[0]: scanning scsi channel 1 [Phy 1] for non-raid devices
input: ImPS/2 Generic Wheel Mouse as /class/input/input1
scsi[0]: scanning scsi channel 2 [virtual] for logical drives
Vendor: MegaRAID Model: LD 0 RAID5 1678G Rev: 351X
Type: Direct-Access ANSI SCSI revision: 02
SCSI device sda: 3438182400 512-byte hdwr sectors (1760349 MB)
sda: Write Protect is off
sda: Mode Sense: 00 00 00 00
sda: asking for cache data failed
sda: assuming drive cache: write through
SCSI device sda: 3438182400 512-byte hdwr sectors (1760349 MB)
sda: Write Protect is off
sda: Mode Sense: 00 00 00 00
sda: asking for cache data failed
sda: assuming drive cache: write through
sda: sda1
sd 0:2:0:0: Attached scsi disk sda
megaraid: probe new device 0x1028:0x0013:0x1028:0x016c: bus 2:slot
14:func 0
ACPI: PCI Interrupt 0000:02:0e.0[A] -> GSI 46 (level, low) -> IRQ 20
megaraid: fw version:[521X] bios version:[H430]
scsi1 : LSI Logic MegaRAID driver
scsi[1]: scanning scsi channel 0 [Phy 0] for non-raid devices
Vendor: PE/PV Model: 1x2 SCSI BP Rev: 1.0
Type: Processor ANSI SCSI revision: 02
scsi[1]: scanning scsi channel 1 [virtual] for logical drives
Vendor: MegaRAID Model: LD 0 RAID1 69G Rev: 521X
Type: Direct-Access ANSI SCSI revision: 02
SCSI device sdb: 143114240 512-byte hdwr sectors (73274 MB)
sdb: Write Protect is off
sdb: Mode Sense: 00 00 00 00
sdb: asking for cache data failed
sdb: assuming drive cache: write through
SCSI device sdb: 143114240 512-byte hdwr sectors (73274 MB)
sdb: Write Protect is off
sdb: Mode Sense: 00 00 00 00
sdb: asking for cache data failed
sdb: assuming drive cache: write through
sdb: sdb1 sdb2 sdb3 sdb4 < sdb5 >
sd 1:1:0:0: Attached scsi disk sdb
_______________________________________________
Linux-PowerEdge mailing list
Linux-PowerEdge at dell.com
http://lists.us.dell.com/mailman/listinfo/linux-poweredge
Please read the FAQ at http://lists.us.dell.com/faq
More information about the Linux-PowerEdge
mailing list