Odd problem with second interface (bnx2) on Dell R710

Carlson, Timothy S Timothy.Carlson at pnl.gov
Mon Jan 4 17:45:28 CST 2010


I've got a number of Dell R710 machines with the standard quad bnx2 internal network card. Now I have a few of these that I am using as cluster login nodes and they have eth0 pointing to the internal cluster network and eth1 pointing to our corporate LAN. A couple of these boxes simply refuse to run eth1 at gigabit speed. They all come up at 100Mb full duplex.

I've gone through the standard troubleshooting procedures (I think):

Update to latest BIOS on R710 (1.3.6)
Update to latest Broadcomm firmware from Dell (NETW_FRMW_LX_R246418.BIN)
Update to latest bnx2 driver from Dell (v1.9.20d)
Swapped out ethernet cables
Double check settings on switch (it is a 1U 48 port Cisco although I have the same problem connecting to a Netgear). 
Done googling around and added
"options bnx2 disable_msi=1" to /etc/modprobe.conf

The box is running RHEL 5.4 and I see this problem with both stock -164 kernel and the older -128.7.1.el5_lustre.1.8.1.1 kernel that I really need to be using. ethtool always reports eth1 as running at 100Mb/s except when the network is first coming up and it has "unknown" values for both Speed and Duplex.

Has anybody else encountered something like this?


# ethtool eth1
Settings for eth1:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 100Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: g
        Wake-on: d
        Link detected: yes

If I try and force eth1 to be 1000Mb/s, I get the following error:

# ethtool -s eth1 speed 1000 autoneg off duplex full
Cannot set new settings: Invalid argument
  not setting speed
  not setting duplex
  not setting autoneg

Any help or hints would be greatly appreciated!

Thanks

Tim

__________________________________________________ 
Tim Carlson
Senior Research Scientist 
EMSL: Environmental Molecular Sciences Laboratory 
Pacific Northwest National Laboratory 
902 Battelle Boulevard 
P.O. Box 999, MSIN K8-83 
Richland, WA  99352 USA 
Tel:  509-371-6435 
tim at pnl.gov 
www.emsl.pnl.gov 





More information about the Linux-PowerEdge mailing list