intro; successful install of Redhat via PXE netbooting; broadcom jumbo frames question
dag at sonsorol.org
Mon Apr 29 16:32:01 CDT 2002
I've been lurking on this list for a while and thought I'd introduce
myself to see if there are any other informatics-types on this list. I
hope this is appropriate.
I'll do the long-winded intro first and then get to my question -
After joining a computational biology group as a 'wanna-be' scientist
about 6 years ago I quickly got far more interested in high performance
research computing and in particular large IT infrastructures that
support computational biology and life science informatics work...I
guess the current buzzword for what I do now is "Bio-IT" given that IDG
now has a trade rag by that name.
Anyway I've spent most of the last year and a half building linux
compute farms for academic, biotech and clients.
I'm working right now on my first "mostly Dell" biocluster for a group
at Harvard University. Pictures from the in-progress build are online at
The system breaks down into these main pieces:
o 30 dual-CPU Poweredge 1550 compute elements (phase I)
o 4 dual-CPU PowerEdge 6450 servers
o NetApp F840 NAS filer with ~3TB usable disk space
o ExtremeNetworks Alpine 3808 GigE core switch
o Dell powerconnect 3024 "edge" switches in the cluster
o SAN-attached tape library for backups
My initial reactions on setting this system up with Dell hardware have
been very very positive.
The best thing so far was how easy it was to boot the 1550's and have
them automatically partition their own disks and autoinstall linux
entirely via the PXE-enabled onboard NIC cards. I'm using SystemImager
(www.systemimager.org) to handle unattended OS installs and incremental
I'm at the point now where I can boot a clean poweredge 1550 box off the
network and ~3 minutes later I've got a fully partitioned, loaded and
functional cluster element. I've been using SystemImager forever and
this is only the 2nd time I've been able to easily get the PXE-based
autoinstall stuff to work so smoothly. I usually use the SI autoinstall
CDROM's to bootstrap new systems.
I'm really happy that this was so easy to do on the poweredge hardware.
Anybody else doing this? Any other systemimager users out there? I may
end up writing a mini-howto on this for the SI FAQ if there is interest.
I have a question relating to jumbo frame support for the copper
Broadcomm gigabit ethernet cards that I have in the 6450 servers. The
NetApp 840, the Alpine 3808 and the broadcom NIC's all claim to support
jumbo frames but each time I crank the MTU on the netapp interface above
1500 bytes I hang my 6450 NFS client (I'm forcing NFSv3 on the client
side if it matters).
I've been adjusting the MTU on the 6450 servers by putting an "option
mtu=8194" directive into /etc/modules.conf and then reloading the kernel
module. A quick check of the info in /proc seems to show that the change
successfully occurs. However- I am totally unable to tweak the MTU above
1500 using the standard "ifconfig <if-dev> mtu=9000" command so I'm not
actually certain that I'm tweaking the module the correct way.
Anyone using jumbo frames successfully for copper GigE NICs with the
broadcomm chipsets? Any pointers to online docs or FAQs would be greatly
Chris Dagdigian, <dag at sonsorol.org>
Independent life science IT & research computing consulting
Office: 617-666-6454, Mobile: 617-877-5498, Fax: 425-699-0193
Work: http://BioTeam.net PGP KeyID: 83D4310E Yahoo IM: craffi
More information about the Linux-PowerEdge