[Linux-PowerEdge] The Ubuntu server installation process breaks iDRAC ethernet comms

Andrew Gaylard a.gaylard at gmail.com
Wed Oct 19 10:47:24 CDT 2016


Running the Ubuntu server install process on a R310 causes its iDRAC6
to lose ethernet communications.  The program in particular which
causes this is netcfg.  I've tested this with 14.04, 16.04, and 16.10.

Naturally, this makes installing via a remote console impossible.

Sometimes a server reboot is sufficient to restore ethernet comms.
However, without iDRAC comms, it's necessary to pay the server a visit
to perform such a reboot.  Occasionally, though, it's even necessary
to remove the power cables for a short period to restore iDRAC

Once Ubuntu is installed and running, the iDRAC unit works fine.

Further digging reveals that this problem can also be triggered by

     /sbin/ip link set up dev em1; /sbin/ip link set down dev em1
     /sbin/ip link set up dev em2; /sbin/ip link set down dev em2

*even* *though* em2 is not physically connected! (em1 is carrying both
the server and iDRAC traffic.)  After doing this, iDRAC comms can
sometimes be restored with "ip link set up ..." on both interfaces.
But sometimes not, and a trip to the data centre across town is

I've tried upgrading the firmware to the latest versions:

      iDRAC6 (Build 4)
      BIOS   1.12.0
      BNX    8.07.25

I've tried running

      uxdiag -T abcd -mfw 0
      uxdiag -T abcd -mfw 1

from DOS; disabling and reenabling replicated the symptoms: the iDRAC6
would lose ether comms on "0" and recover on "1".

I've tried setting the iDRAC's ethernet to 100/full (not auto-
negotiate) using "racadm config -g cfgNetTuning ...".

However, none of these things made any difference.  Ubuntu's installer
still causes loss of iDRAC ether comms.

The Linux kernel reports thusly during boot; when iDRAC comms is lost,
the server's traffic continues to flow as normal and the kernel
reports nothing new.

[    3.478431] bnx2: QLogic bnx2 Gigabit Ethernet Driver v2.2.6 (January 
29, 2014)
[    3.486582] bnx2 0000:02:00.0 eth0: Broadcom NetXtreme II BCM5716 
1000Base-T (C0) PCI Express found at mem d8000000, IRQ 16, node addr 
[    3.500461] md: multipath personality registered for level -4
[    3.501034] bnx2 0000:02:00.1 eth1: Broadcom NetXtreme II BCM5716 
1000Base-T (C0) PCI Express found at mem dc000000, IRQ 17, node addr 
[    3.528520] bnx2 0000:02:00.1 em2: renamed from eth1
[    3.553714] bnx2 0000:02:00.0 em1: renamed from eth0
[   18.031089] bnx2 0000:02:00.0 em1: using MSIX
[   20.067736] bnx2 0000:02:00.0 em1: NIC Copper Link is Up, 100 Mbps 
full duplex

Any ideas?

Andrew Gaylard

More information about the Linux-PowerEdge mailing list