Odd problem with second interface (bnx2) on Dell R710
Carlson, Timothy S
Timothy.Carlson at pnl.gov
Mon Jan 4 17:45:28 CST 2010
I've got a number of Dell R710 machines with the standard quad bnx2 internal network card. Now I have a few of these that I am using as cluster login nodes and they have eth0 pointing to the internal cluster network and eth1 pointing to our corporate LAN. A couple of these boxes simply refuse to run eth1 at gigabit speed. They all come up at 100Mb full duplex.
I've gone through the standard troubleshooting procedures (I think):
Update to latest BIOS on R710 (1.3.6)
Update to latest Broadcomm firmware from Dell (NETW_FRMW_LX_R246418.BIN)
Update to latest bnx2 driver from Dell (v1.9.20d)
Swapped out ethernet cables
Double check settings on switch (it is a 1U 48 port Cisco although I have the same problem connecting to a Netgear).
Done googling around and added
"options bnx2 disable_msi=1" to /etc/modprobe.conf
The box is running RHEL 5.4 and I see this problem with both stock -164 kernel and the older -128.7.1.el5_lustre.126.96.36.199 kernel that I really need to be using. ethtool always reports eth1 as running at 100Mb/s except when the network is first coming up and it has "unknown" values for both Speed and Duplex.
Has anybody else encountered something like this?
# ethtool eth1
Settings for eth1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
Advertised auto-negotiation: Yes
Port: Twisted Pair
Supports Wake-on: g
Link detected: yes
If I try and force eth1 to be 1000Mb/s, I get the following error:
# ethtool -s eth1 speed 1000 autoneg off duplex full
Cannot set new settings: Invalid argument
not setting speed
not setting duplex
not setting autoneg
Any help or hints would be greatly appreciated!
Senior Research Scientist
EMSL: Environmental Molecular Sciences Laboratory
Pacific Northwest National Laboratory
902 Battelle Boulevard
P.O. Box 999, MSIN K8-83
Richland, WA 99352 USA
tim at pnl.gov
More information about the Linux-PowerEdge