Redundant NFS storage setup (part 2)

[Diego_Leccardi at Dell.com]

Hi Matthias,

> I would like to go with ext3, but I'll hit the 8TB filesystem size limit...

You can used RHEL 5.1, it has support for 16 TB ext3 fs.

RHEL 5.0 included a TechPreview of ext3 up to 16TB. As of RHEL 5.1, this is fully supported. 
Have a look at the Release Notes:
http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/release-notes/RELEASE-NOTES-U1-x86_64-en.html
or
http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/release-notes/RELEASE-NOTES-U1-x86-en.html

To create a 16TB ext3 filesystem, you'll need to run mke2fs with the -F flag.

Hope this help

Diego

-----Original Message-----
From: linux-poweredge-bounces at dell.com [mailto:linux-poweredge-bounces at dell.com] On Behalf Of Matthias Saou
Sent: 23 November 2007 10:11
To: linux-poweredge-Lists
Subject: Redundant NFS storage setup (part 2)

Hi,

A quick followup on my file server project exposed previously :
http://lists.us.dell.com/pipermail/linux-poweredge/2007-November/033616.html

First of all, thanks for all the answers! Some tips were _very_ useful.

Here's what I've decided to go with :
- Two PE1950 with lots of RAM, CPU, disk, network interfaces (bonding)
- Two MD1000 with a single EMM and 15 1TB drives

Each PowerEdge will be connected to its own PowerVault MD1000, and
access a single huge filesystem. I would like to go with ext3, but
I'll hit the 8TB filesystem size limit... so I'll try to hack around
the problem by creating two filesystems on two LVM LVs and mount both
separately. A Dell guy told me that with SAS and the PERC5/E it should
be possible to configure a single RAID-5 array greater than 2TB. I hope
this is true, otherwise I'll have to do the same old ugly hacks as with
SCSI and concatenate PVs with LVM.

This will give a two-node hardware redundant setup. And it's "cheap"
compared to any "higher end" setup, like MD3000s, EMC² devices etc.

On the software side, I'll be mostly on my own as usual. But it's what I
want anyway, using only free software ;-) The OS will be RHEL5 and the
file synchronization will be achieved using csync2. To protect from
accidental data loss being also synced, some local incremental backups
will be made on both servers, using hardlinks and set immutable during
normal operation.

The tricky part is serving all of the files to multiple clients in a
load-balanced manner, with automatic failover. Well, from my tests,
glusterfs is the answer! It's really, REALLY, neat. Think of a simple
and much more flexible NFS ;-) It's supposed to be able to cluster
nodes in an HA way, but that fails miserably in my tests with 1.3.7,
hence the decision to use csync2. The load-balancing is achieved simply
by telling clients to mount the share from a DNS address pointing to
two IP addresses. They'll initially connect randomly to either server.
If one server goes down, it takes less than 5s for the client to try
to reconnect and get the other server. Dead simple. The downside is
that when the failed node comes back online, clients will not reconnect
to it automatically. This should be easy to solve by using floating IP
addresses and keepalived (VRRP) or UCARP.

The bit where I'm really lucky is that all of the files will be served
read-only, so no locking, concurrent access or replication conflict
problems... it should "just work".

Here are all the files I use for RHEL5 :
http://ftp.es6.freshrpms.net/pub/freshrpms/redhat/testing/EL5/cluster/
(glusterfs and csync2 packages include major improvements)

Pointers :
http://oss.linbit.com/csync2/
http://www.gluster.org/docs/index.php/GlusterFS

Comments and suggestions are welcome.

Matthias

-- 
Clean custom Red Hat Linux rpm packages : http://freshrpms.net/
Fedora release 8 (Werewolf) - Linux kernel 2.6.23.1-49.fc8
Load : 0.25 0.24 0.26

_______________________________________________
Linux-PowerEdge mailing list
Linux-PowerEdge at dell.com
http://lists.us.dell.com/mailman/listinfo/linux-poweredge
Please read the FAQ at http://lists.us.dell.com/faq