[Linux-PowerEdge] omreport suddenly not giving information

Patrick Boutilier boutilpj at ednet.ns.ca
Wed Feb 15 11:05:34 CST 2017


Feb 15 16:17:25 albion BMAPI[34316]: ERROR       SemCreate() semget() 
failed! No space left on device

Looks like you ran out of semaphores. Couple of ways to verify:

ipcs -s (this should show you the current semaphore arrays in use)


These two show current settings:

ipcs -sl
sysctl kernel.sem


Increase value of kernel.sem in /etc/sysctl.conf and then run:

sysctl -p



On 02/15/2017 12:37 PM, Ben Argyle wrote:
> I've got a 720xd (revision I) running RHEL6 and OMSA 8.4.0 with the following firmware:
>
> BIOS	2.5.4
> iDRAC	2.32.31.30
>
> About half an hour ago it started not giving any output for "omreport chassis".  I restarted OMSA with "/opt/dell/srvadmin/sbin/srvadm-service.sh restart" but still got no joy.  I logged into the DRAC via SSH and did "racadm racreset soft", waited for it to come back and then tried again (after restarting OMSA services again).  No joy.  But now, in addition, "omreport system" and "omreport storage controller" also don't return information.  In the latter case omreport states "No controllers found".
>
> Can anyone tell me what's gone wrong?  The OS is still working perfectly, and the DRAC GUI _seems_ to be giving me all the usual information, but omreport within the OS isn't.
>
> In addition dsu hangs when doing this:
>
> # dsu --inventory
> Verifying catalog installation ...
> Installing catalog from repository ...
> Fetching dsucatalog ...
> Reading the catalog ...
> Installing inventory collector ...
> Fetching invcol_WF06C_LN64_16.12.200.896_A00 ...
> Verifying inventory collector installation ...
> Getting System Inventory ...
>
> Below is /var/log/messages from when I did the first "srvadm-services.sh restart".  There are no errors or warnings above it.  This server has a lot of FC mounts and unmounts happening on it.  Any thoughts?
>
> Feb 15 15:49:58 albion dataeng: dsm_sa_snmpd shutdown succeeded
> Feb 15 15:50:00 albion dataeng: dsm_sa_eventmgrd shutdown succeeded
> Feb 15 15:50:08 albion dataeng: dsm_sa_datamgrd shutdown succeeded
> Feb 15 15:50:09 albion instsvcdrv: dell_rbu device driver unloaded
> Feb 15 15:50:09 albion instsvcdrv: dell_rbu device driver loaded
> Feb 15 15:50:20 albion dataeng: warning: snmpd not started. snmpd must be started to manage this system using SNMP.
> Feb 15 16:01:46 albion kernel: usb 1-1.6: USB disconnect, device number 4
> Feb 15 16:01:46 albion kernel: usb 1-1.6.1: USB disconnect, device number 5
> Feb 15 16:01:56 albion kernel: usb 1-1.6: new high-speed USB device number 6 using ehci-pci
> Feb 15 16:01:57 albion kernel: usb 1-1.6: New USB device found, idVendor=413c, idProduct=a001
> Feb 15 16:01:57 albion kernel: usb 1-1.6: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> Feb 15 16:01:57 albion kernel: usb 1-1.6: Product: Gadget USB HUB
> Feb 15 16:01:57 albion kernel: usb 1-1.6: Manufacturer: no manufacturer
> Feb 15 16:01:57 albion kernel: usb 1-1.6: SerialNumber: 0123456789
> Feb 15 16:01:57 albion kernel: hub 1-1.6:1.0: USB hub found
> Feb 15 16:01:57 albion kernel: hub 1-1.6:1.0: 6 ports detected
> Feb 15 16:03:00 albion kernel: usb 1-1.6.1: new high-speed USB device number 7 using ehci-pci
> Feb 15 16:03:00 albion kernel: usb 1-1.6.1: New USB device found, idVendor=0624, idProduct=0249
> Feb 15 16:03:00 albion kernel: usb 1-1.6.1: New USB device strings: Mfr=4, Product=5, SerialNumber=6
> Feb 15 16:03:00 albion kernel: usb 1-1.6.1: Product: Keyboard/Mouse Function
> Feb 15 16:03:00 albion kernel: usb 1-1.6.1: Manufacturer: Avocent
> Feb 15 16:03:00 albion kernel: usb 1-1.6.1: SerialNumber: 20121018
> Feb 15 16:03:00 albion kernel: input: Avocent Keyboard/Mouse Function as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.1/1-1.6.1:1.0/input/input5
> Feb 15 16:03:00 albion kernel: hid-generic 0003:0624:0249.0004: input,hidraw0: USB HID v1.00 Keyboard [Avocent Keyboard/Mouse Function] on usb-0000:00:1a.0-1.6.1/input0
> Feb 15 16:03:00 albion kernel: input: Avocent Keyboard/Mouse Function as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.1/1-1.6.1:1.1/input/input6
> Feb 15 16:03:00 albion kernel: hid-generic 0003:0624:0249.0005: input,hidraw1: USB HID v1.00 Mouse [Avocent Keyboard/Mouse Function] on usb-0000:00:1a.0-1.6.1/input1
> Feb 15 16:03:00 albion kernel: input: Avocent Keyboard/Mouse Function as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.1/1-1.6.1:1.2/input/input7
> Feb 15 16:03:00 albion kernel: hid-generic 0003:0624:0249.0006: input,hidraw2: USB HID v1.00 Mouse [Avocent Keyboard/Mouse Function] on usb-0000:00:1a.0-1.6.1/input2
> Feb 15 16:03:01 albion kernel: usb 1-1.6.3: new high-speed USB device number 8 using ehci-pci
> Feb 15 16:03:01 albion kernel: usb 1-1.6.3: New USB device found, idVendor=413c, idProduct=a102
> Feb 15 16:03:01 albion kernel: usb 1-1.6.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
> Feb 15 16:03:01 albion kernel: usb 1-1.6.3: Product: iDRAC Virtual NIC USB Device
> Feb 15 16:03:01 albion kernel: usb 1-1.6.3: Manufacturer: Dell(TM)
> Feb 15 16:03:01 albion kernel: cdc_ether 1-1.6.3:1.0 usb0: register 'cdc_ether' at usb-0000:00:1a.0-1.6.3, CDC Ethernet Device, ce:7f:94:9c:6a:22
> Feb 15 16:03:01 albion kernel: usbcore: registered new interface driver cdc_ether
> Feb 15 16:03:01 albion kernel: net usb0: 'usb0' renaming to 'idrac'
> Feb 15 16:03:06 albion kernel: usb 1-1.6.3: USB disconnect, device number 8
> Feb 15 16:03:06 albion kernel: cdc_ether 1-1.6.3:1.0 idrac: unregister 'cdc_ether' usb-0000:00:1a.0-1.6.3, CDC Ethernet Device
> Feb 15 16:03:20 albion kernel: usb 1-1.6.1: USB disconnect, device number 7
> Feb 15 16:03:21 albion kernel: usb 1-1.6.1: new high-speed USB device number 9 using ehci-pci
> Feb 15 16:03:21 albion kernel: usb 1-1.6.1: New USB device found, idVendor=0624, idProduct=0249
> Feb 15 16:03:21 albion kernel: usb 1-1.6.1: New USB device strings: Mfr=4, Product=5, SerialNumber=6
> Feb 15 16:03:21 albion kernel: usb 1-1.6.1: Product: Keyboard/Mouse Function
> Feb 15 16:03:21 albion kernel: usb 1-1.6.1: Manufacturer: Avocent
> Feb 15 16:03:21 albion kernel: usb 1-1.6.1: SerialNumber: 20121018
> Feb 15 16:03:21 albion kernel: input: Avocent Keyboard/Mouse Function as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.1/1-1.6.1:1.0/input/input8
> Feb 15 16:03:21 albion kernel: hid-generic 0003:0624:0249.0007: input,hidraw0: USB HID v1.00 Keyboard [Avocent Keyboard/Mouse Function] on usb-0000:00:1a.0-1.6.1/input0
> Feb 15 16:03:21 albion kernel: input: Avocent Keyboard/Mouse Function as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.1/1-1.6.1:1.1/input/input9
> Feb 15 16:03:21 albion kernel: hid-generic 0003:0624:0249.0008: input,hidraw1: USB HID v1.00 Mouse [Avocent Keyboard/Mouse Function] on usb-0000:00:1a.0-1.6.1/input1
> Feb 15 16:03:21 albion kernel: input: Avocent Keyboard/Mouse Function as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.1/1-1.6.1:1.2/input/input10
> Feb 15 16:03:21 albion kernel: hid-generic 0003:0624:0249.0009: input,hidraw2: USB HID v1.00 Mouse [Avocent Keyboard/Mouse Function] on usb-0000:00:1a.0-1.6.1/input2
> Feb 15 16:04:21 albion dataeng: dsm_sa_snmpd shutdown succeeded
> Feb 15 16:04:22 albion dataeng: dsm_sa_eventmgrd shutdown succeeded
> Feb 15 16:04:29 albion dataeng: dsm_sa_datamgrd shutdown succeeded
> Feb 15 16:04:30 albion instsvcdrv: dell_rbu device driver unloaded
> Feb 15 16:04:30 albion instsvcdrv: dell_rbu device driver loaded
> Feb 15 16:04:49 albion dataeng: warning: snmpd not started. snmpd must be started to manage this system using SNMP.
> Feb 15 16:15:23 albion yum[25501]: Erased: dsucatalog
> Feb 15 16:15:39 albion yum[25512]: Installed: dsucatalog-17.01.00-TDDR9.noarch
> Feb 15 16:15:58 albion yum[25576]: Installed: invcol_WF06C_LN64_16.12.200.896_A00-16.12.200.896-WF06C.x86_64
> Feb 15 16:16:14 albion kernel: Initializing USB Mass Storage driver...
> Feb 15 16:16:14 albion kernel: usbcore: registered new interface driver usb-storage
> Feb 15 16:16:14 albion kernel: USB Mass Storage support registered.
> Feb 15 16:16:16 albion kernel: usb 1-1.6.2: new high-speed USB device number 10 using ehci-pci
> Feb 15 16:16:16 albion kernel: usb 1-1.6.2: New USB device found, idVendor=0624, idProduct=0250
> Feb 15 16:16:16 albion kernel: usb 1-1.6.2: New USB device strings: Mfr=4, Product=5, SerialNumber=6
> Feb 15 16:16:16 albion kernel: usb 1-1.6.2: Product: Mass Storage Function
> Feb 15 16:16:16 albion kernel: usb 1-1.6.2: Manufacturer: Avocent
> Feb 15 16:16:16 albion kernel: usb 1-1.6.2: SerialNumber: 20120731
> Feb 15 16:16:16 albion kernel: scsi3 : usb-storage 1-1.6.2:1.0
> Feb 15 16:16:17 albion kernel: scsi 3:0:0:0: Direct-Access     iDRAC    SECUPD           0329 PQ: 0 ANSI: 0 CCS
> Feb 15 16:16:17 albion kernel: sd 3:0:0:0: Attached scsi generic sg40 type 0
> Feb 15 16:16:17 albion kernel: sd 3:0:0:0: [sdao] 2112 512-byte logical blocks: (1.08 MB/1.03 MiB)
> Feb 15 16:16:17 albion kernel: sd 3:0:0:0: [sdao] Write Protect is off
> Feb 15 16:16:17 albion kernel: sd 3:0:0:0: [sdao] No Caching mode page found
> Feb 15 16:16:17 albion kernel: sd 3:0:0:0: [sdao] Assuming drive cache: write through
> Feb 15 16:16:18 albion kernel: sd 3:0:0:0: [sdao] No Caching mode page found
> Feb 15 16:16:18 albion kernel: sd 3:0:0:0: [sdao] Assuming drive cache: write through
> Feb 15 16:16:18 albion kernel: sdao:
> Feb 15 16:16:18 albion kernel: sd 3:0:0:0: [sdao] No Caching mode page found
> Feb 15 16:16:18 albion kernel: sd 3:0:0:0: [sdao] Assuming drive cache: write through
> Feb 15 16:16:18 albion kernel: sd 3:0:0:0: [sdao] Attached SCSI removable disk
> Feb 15 16:16:18 albion multipathd: sdao: add path (uevent)
> Feb 15 16:16:18 albion multipathd: sdao: failed to get path uid
> Feb 15 16:16:18 albion multipathd: uevent trigger error
> Feb 15 16:16:34 albion kernel: usb 1-1.6.2: USB disconnect, device number 10
> Feb 15 16:16:34 albion multipathd: sdao: remove path (uevent)
> Feb 15 16:16:39 albion kernel: usb 1-1.6.2: new high-speed USB device number 11 using ehci-pci
> Feb 15 16:16:39 albion kernel: usb 1-1.6.2: New USB device found, idVendor=0624, idProduct=0250
> Feb 15 16:16:39 albion kernel: usb 1-1.6.2: New USB device strings: Mfr=4, Product=5, SerialNumber=6
> Feb 15 16:16:39 albion kernel: usb 1-1.6.2: Product: Mass Storage Function
> Feb 15 16:16:39 albion kernel: usb 1-1.6.2: Manufacturer: Avocent
> Feb 15 16:16:40 albion kernel: usb 1-1.6.2: SerialNumber: 20120731
> Feb 15 16:16:40 albion kernel: scsi4 : usb-storage 1-1.6.2:1.0
> Feb 15 16:16:41 albion kernel: scsi 4:0:0:0: Direct-Access     iDRAC    SECUPD           0329 PQ: 0 ANSI: 0 CCS
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: Attached scsi generic sg40 type 0
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: [sdao] 2112 512-byte logical blocks: (1.08 MB/1.03 MiB)
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: [sdao] Write Protect is off
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: [sdao] No Caching mode page found
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: [sdao] Assuming drive cache: write through
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: [sdao] No Caching mode page found
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: [sdao] Assuming drive cache: write through
> Feb 15 16:16:41 albion kernel: sdao:
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: [sdao] No Caching mode page found
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: [sdao] Assuming drive cache: write through
> Feb 15 16:16:41 albion kernel: sd 4:0:0:0: [sdao] Attached SCSI removable disk
> Feb 15 16:16:41 albion multipathd: sdao: add path (uevent)
> Feb 15 16:16:41 albion multipathd: sdao: failed to get path uid
> Feb 15 16:16:41 albion multipathd: uevent trigger error
> Feb 15 16:16:56 albion kernel: usb 1-1.6.2: USB disconnect, device number 11
> Feb 15 16:16:56 albion multipathd: sdao: remove path (uevent)
> Feb 15 16:16:59 albion kernel: usbcore: deregistering interface driver usb-storage
> Feb 15 16:17:07 albion kernel: dchcfg[31998]: segfault at 0 ip 000000370813382f sp 00007ffd1346ad88 error 4 in libc-2.12.so[3708000000+18a000]
> Feb 15 16:17:07 albion abrt[32005]: Saved core dump of pid 31998 (/opt/dell/dup64/sbin/dchcfg) to /var/spool/abrt/ccpp-2017-02-15-16:17:07-31998 (454656 bytes)
> Feb 15 16:17:07 albion kernel: dchcfg[32046]: segfault at 0 ip 000000370813382f sp 00007ffe9dd0ad68 error 4 in libc-2.12.so[3708000000+18a000]
> Feb 15 16:17:07 albion abrt[32052]: Not saving repeating crash in '/opt/dell/dup64/sbin/dchcfg'
> Feb 15 16:17:07 albion kernel: dchcfg[32090]: segfault at 0 ip 000000370813382f sp 00007fffb917b1e8 error 4 in libc-2.12.so[3708000000+18a000]
> Feb 15 16:17:07 albion abrt[32098]: Not saving repeating crash in '/opt/dell/dup64/sbin/dchcfg'
> Feb 15 16:17:07 albion abrtd: Directory 'ccpp-2017-02-15-16:17:07-31998' creation detected
> Feb 15 16:17:07 albion abrtd: Executable '/opt/dell/dup64/sbin/dchcfg' doesn't belong to any package and ProcessUnpackaged is set to 'no'
> Feb 15 16:17:07 albion abrtd: 'post-create' on '/var/spool/abrt/ccpp-2017-02-15-16:17:07-31998' exited with 1
> Feb 15 16:17:07 albion abrtd: Deleting problem directory '/var/spool/abrt/ccpp-2017-02-15-16:17:07-31998'
> Feb 15 16:17:15 albion kernel: dchcfg[33467]: segfault at 0 ip 000000370813382f sp 00007ffc87e3ca98 error 4 in libc-2.12.so[3708000000+18a000]
> Feb 15 16:17:15 albion abrt[33468]: Not saving repeating crash in '/opt/dell/dup64/sbin/dchcfg'
> Feb 15 16:17:15 albion kernel: dchcfg[33470]: segfault at 0 ip 000000370813382f sp 00007ffe7e7084e8 error 4 in libc-2.12.so[3708000000+18a000]
> Feb 15 16:17:15 albion abrt[33471]: Not saving repeating crash in '/opt/dell/dup64/sbin/dchcfg'
> Feb 15 16:17:15 albion kernel: dchcfg[33497]: segfault at 0 ip 000000370813382f sp 00007ffe1a7c1238 error 4 in libc-2.12.so[3708000000+18a000]
> Feb 15 16:17:15 albion abrt[33498]: Not saving repeating crash in '/opt/dell/dup64/sbin/dchcfg'
> Feb 15 16:17:15 albion kernel: dchcfg[33573]: segfault at 0 ip 000000370813382f sp 00007ffea292cf18 error 4 in libc-2.12.so[3708000000+18a000]
> Feb 15 16:17:15 albion abrt[33574]: Not saving repeating crash in '/opt/dell/dup64/sbin/dchcfg'
> Feb 15 16:17:15 albion kernel: dchcfg[33655]: segfault at 0 ip 000000370813382f sp 00007ffec47fd148 error 4 in libc-2.12.so[3708000000+18a000]
> Feb 15 16:17:15 albion abrt[33656]: Not saving repeating crash in '/opt/dell/dup64/sbin/dchcfg'
> Feb 15 16:17:25 albion BMAPI[34316]: ERROR       SemCreate() semget() failed! No space left on device
> Feb 15 16:17:25 albion BMAPI[34316]: ERROR       BmapiInitialize() LockCreate() failed!
> Feb 15 16:17:25 albion BMAPI[34316]: ERROR       BmapiInitialize() LockCreate() failed!
>
> Ben
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: boutilpj.vcf
Type: text/x-vcard
Size: 286 bytes
Desc: not available
Url : http://lists.us.dell.com/pipermail/linux-poweredge/attachments/20170215/b7ffdd88/attachment-0001.vcf 


More information about the Linux-PowerEdge mailing list