Dell 2650 e1000 and tg3 bonding issues

Fri Dec 1 18:54:29 CST 2006

Your problem almost certainly lies in the fact that the bonding documentation in the 2.6.9 kernel is out of date with respect to the module loading mechanisms in the 2.6.9 kernel. The "options" lines are no longer a viable means to configure multiple bonds with different properties. The complete configuration solution should be:

alias eth0 e1000
alias eth1 e1000
alias eth2 tg3
alias eth3 tg3

install bond0 modprobe bonding --ignore-install -o bond0 \
	mode=<mode> [additional options]
install bond1 modprobe bonding --ignore-install -o bond1 \
	mode=<mode> [additional options]

Note that `alias' lines are not necessary for the bonds and should not be used. Make sure to unload any existing bonding instances before testing this solution.

Thomas

-----Original Message-----
From: linux-poweredge-bounces at dell.com [mailto:linux-poweredge-bounces at dell.com] On Behalf Of Aaron
Sent: Friday, December 01, 2006 2:48 PM
To: linux-poweredge-Lists
Subject: Dell 2650 e1000 and tg3 bonding issues

PROBLEM STATEMENT:
   Regardless of configuration, the bonding driver does 
not adhere to the options given.
   Dozens of modprobe.conf incantations have been 
performed and yet, the demon has not been exercised.

   The information in bond0 and bond1 indicate that 
options are being ignored in modprobe.conf.

   You can ignore the link failures.  I was performing 
failover testing.

OS: CentOS 4.3

SERVER: Dell 2650 with one pci Intel-1000MT Dual port gig 
nic and 2 on-board broadcom nics.
         BIOS is A10 which is old but does not seem to be 
related to this issue.

NETWORK: 2 Front end connections to Cisco 3550 100Mb 
interfaces (Fairly up to date IOS).
          2 Back end connections to Cisco 6506 (Latest 
IOS)

cat /proc/net/bonding/bond0 /proc/net/bonding/bond1

Ethernet Channel Bonding Driver: v2.6.1 (October 29, 2004)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: eth1
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 2
Permanent HW addr: 00:04:23:d6:e7:a8

Slave Interface: eth2
MII Status: up
Link Failure Count: 3
Permanent HW addr: 00:06:5b:fe:55:cb

Ethernet Channel Bonding Driver: v2.6.1 (October 29, 2004)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: eth1
Currently Active Slave: eth1
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth1
MII Status: up
Link Failure Count: 2
Permanent HW addr: 00:04:23:d6:e7:a9

Slave Interface: eth3
MII Status: up
Link Failure Count: 2
Permanent HW addr: 00:06:5b:fe:55:cc

I read 
/usr/share/doc/kernel-doc-2.6.9/Documentation/networking/bonding.txt 
forwards and backwards and upside down with a security 
guard staring at me.

I have tried setting the bond/nic options in many 
different orders.

I have not specified any ethtool options, as I have not 
had any problems negotiating to the correct speed and 
duplex.  I am tempted to try that however, just for fun.

## modprobe.conf
   ( tried many variations of this and everything in 
bonding.txt                )
   ( tried commenting out the individual bond opts, 
leaving out the install, etc)

install bonding /sbin/modprobe e1000;/sbin/modprobe 
tg3;/sbin/modprobe --ignore-install bonding max_bonds=2 
mode=1 arp_interval=300 use_carrier=0 -o bond0 
arp_ip_target=10.0.160.1,10.0.160.2,10.0.160.3 mode=1 
use_carrier=0 primary=eth0 -o bonding1 mode=1 
use_carrier=0 primary=eth1 arp_interval=300 
arp_ip_target=192.168.210.100,192.168.210.101,192.168.2,192.168.210.3
#
alias bond0 bonding
options bond0 max_bonds=2 -o bond0 mode=1 use_carrier=0 
primary=eth0 arp_interval=300 
arp_ip_target=10.0.160.1,10.0.160.2,10.0.160.3
alias bond1 bonding
options bond1 -o bonding1 mode=1 use_carrier=0 
primary=eth1 arp_interval=300 
arp_ip_target=192.168.210.100,192.168.210.101,192.168.210.102
alias eth0 e1000
alias eth1 e1000
alias eth2 tg3
alias eth3 tg3
alias scsi_hostadapter aic7xxx
alias usb-controller ohci-hcd

## lspci

01:08.0 Ethernet controller: Intel Corporation 82546GB 
Gigabit Ethernet Controller (rev 03)
01:08.1 Ethernet controller: Intel Corporation 82546GB 
Gigabit Ethernet Controller (rev 03)
03:06.0 Ethernet controller: Broadcom Corporation 
NetXtreme BCM5701 Gigabit Ethernet (rev 15)
03:08.0 Ethernet controller: Broadcom Corporation 
NetXtreme BCM5701 Gigabit Ethernet (rev 15

eth0 (e1000) and eth2 (tg3) are the inside interfaces .:. 
eth1 (e1000) and eth3 (tg3) are the outside interfaces.

Hopefully I have not left any useful information off of 
here.

Any thoughts as to what I am missing?  Anyone else bonding 
a front and back side to a Dell 2650 using tg3 and e1000? 
 At first this looked allot like a problem I had with a 
Compaq DL380G3, but it is different.

--Aarön

_______________________________________________
Linux-PowerEdge mailing list
Linux-PowerEdge at dell.com
http://lists.us.dell.com/mailman/listinfo/linux-poweredge
Please read the FAQ at http://lists.us.dell.com/faq