Slightly OT: Distributed filesystems

[Karl Katzke]

Eric/Christian - We've used OCFS2 on a FC SAN volume. One of the
operations on the shared volume was sparse filesystem image files for
the Xen VMs in an effort to ease the locking issues you normally see
with hot migration. With OCFS2, drive system performance would quickly
plateau. Operations like a full-cluster cold boot were wrenchingly slow
after we hit a certain number of VMs.  

We're switching to clvmd-managed logical volumes for our VMs and then
NFS4 for our shared volume needs, all on SLES11 managed with Pacemaker
and OpenAIS. So far, our performance tests indicate magnitudes of
improvement, with all VMs booting within seconds. That NEVER happened on
OCFS2, which chewed CPU like nobody's business.    

If you still need a distributed filesystem, pNFS / NFS 4.1 has a
distributed mode that is still under development, but is defined in
several draft or completed RFCs (lost track of their status since I
reviewed it a year ago) and already has client/server vendor support
from several high-end storage vendors.  

I have no experience with GFS, but we're running pacemaker/openais in
the default multicast and it's not broken the existing things running in
either broadcast or in multicast on other addresses, so if you figure
out what your Oracle RAC multicast address is and set the openais config
for a different address, you should be fine.  

-K 


---
Karl Katzke
Systems Analyst II
TAMU - DRGS






>>> Christian Rishøj<christian at jetment.com> 5/9/2009 5:45 AM >>>



Hi, 



In my previous experience, OCFS2 (on DRBD shared storage) involved a
rather substantial overhead. I suggest testing such a setup with a
realistic load before putting it into production. 



Also, consider if the added complexity of introducing a distributed
filesystem (with its distributed lock manager, quorum service et al) is
really worth what you gain.  



Best regards, 



Christian 





On 8 May 2009, at 18:47, Eric.Jorgensen at westernunion.com wrote: 






Hello,

I have several 2950s grouped together in Oracle RAC clusters with a SAN
used for shared storage.  I'd like to setup another LUN from the SAN and
make it available on all nodes of a cluster.  I realize that Oracle RAC
has this functionality already, but we are using ASM and I'd like the
files to be readable outside of Oracle (i.e. to Netbackup), so I believe
that using Oracle's tools will not work.

Moreover, I've been looking at Redhat's Global Filesystem, but it
appears that hardware is needed for fencing.  This seems to me to be
overkill, as we already have Oracle handling all the clustering, and I'd
rather not piggyback on the private NICs for example, for fear of
breaking Oracle RAC.

So what I'm looking for is a filesystem I can use to mount the LUN on
each node.  Any pointers would be appreciated, especially any experience
with GFS with non-hardware fencing, OCFS2 without ASM, etc.

Thanks,

Eric




Eric Jorgensen
Sr. Systems Administrator, WU Corporate Systems
Office:  720-332-5615  Ext:  65615
Cell:  303-589-6630   FAX:  720-332-7627
Email:  eric.jorgensen at westernunion.com
12500 E Belford Ave M12SD  Englewood CO 80112 







_______________________________________________
Linux-PowerEdge mailing list
Linux-PowerEdge at dell.com
http://lists.us.dell.com/mailman/listinfo/linux-poweredge
Please read the FAQ at http://lists.us.dell.com/faq  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.us.dell.com/pipermail/linux-poweredge/attachments/20090510/0e4055ad/attachment.htm