nfs performance of 2800.

Jonathan Dill jonathan at nerds.net
Tue Jan 9 22:36:52 CST 2007


Ian Mortimer wrote:
> I've been testing nfs performance of a 2800 (RHEL4 x86_64, PERC 4e/Di,
> Raid 5).  One simple test (bash loop appending lines to a text file)
> gets these timings:
>
>    to 2800 over nfs:                     20.072s
>
>    to old file server over nfs:           4.437s
>
>    on 2800 to local disk:                 0.438s
>
>    on old file server to local disk:      0.464s
>
> (Old file server is 32 bit, 2.4 series kernel)
>
> The surprising result is the far worse nfs performance of the 2800
> compared to the old file server.   Experimenting with mount options
> (rsize, wsize) makes no difference.  nfsvers=2 makes it much worse!
>
> Other tests - writing files with dd, copying or reading large files 
> - get expected results:  the 2800 over nfs is never slower and in 
> some cases more than twice as fast.
>
> Any ideas?
>   
Maybe stupid question, but are the 2800 and other servers that you are 
testing plugged into the same switch?  You could also be testing the 
performance of your switch(es) especially if there is a gigabit to 
slower speed transition somewhere.  The most reliable test is to use a 
crossover cable between the systems that you are using for the test, 
that will rule in or rule out the switches as part of the issue and give 
you a measure of the optimal performance that you can expect.  Granted, 
may not be possible with only production servers around, but you could 
boot a live CD on a reasonable PC and use that as the test client.

try bursts of flood pings from the 2800 to test host to see if you have 
packet loss, try different packet payload sizes to see if there is a 
breaking point where performance becomes much worse

try a smaller wsize / rsize (1024 / 512) and see what happens

try TCP vs. UDP

check ifconfig for MTU and errors, nfsstat, on both server and client

check mii-tool or ethtool to see if Jumbo Frames are enabled and how 
that compares with your network infrastructure, check for duplex mismatch

sniff with wireshark on the connection, check out fragmentation, any 
errors or RSTs

if you have an add-in NIC in the 2800, try removing it and testing over 
the built-in Broadcom

if you are using the on-board Broadcom, chances are you don't have a PCI 
Express NIC laying around, but you could try a USB2 or Firewire to Fast 
Ethernet adapter if you have one of those, would expect to see poorer 
performance, but you might still learn something

Jonathan



More information about the Linux-PowerEdge mailing list