10TB filesystem possible on Red Hat 4 EL System?

[Sam Flory]

Peter Kjellstrom wrote:
> On Wednesday 15 August 2007, Sam Flory wrote:
>> Jack Murgia wrote:
> ...
>>> Do we need to move to RH or CentOS 5 in order to have a 10TB array?
> 
> What do you mean move to RH? You're on RHEL4, right? As for Centos, the only 
> difference is the existence of xfs kernel module and xfsprogs, but those 
> pacakges could be used on RHEL too.
> 
>>   CentOS 5 x86_64 will make things a whole lot easier.  You can then
>> either install the xfs module, or the kernel with the xfsprogs rpm from
>> the Centos extras tree.
> 
> Not quite true nor complete, the xfs stuff lives in centosplus 

  Yup I was confusing Centos terminology with old fedora terminology.

> and you will 
> not need Centos-5 for this. Centos-4 would work too. 

  Yes, and no.  As the xfs code doesn't get update in the RHEL/Centos
kernel.  The Centos4 is about 3 years behind on bug fixes, and other
updates.  Centos-4 will indeed work for most people, but why risk it.

> x86_64, however, would 
> be very close to required for xfs since the 32-bit dist uses 4k kernel stacks 
> which in turn does not play well with xfs...

  Thus why I recommended a 64 bit kernel.

>>   Ext3 can in theory work on a 10TB file system,
> 
> Not on RHEL4 (which is what the OP has). Read:
>  http://www.redhat.com/rhel/compare/

  I only said in theory ext3 can work on a 10TB.

> Only RHEL5 x86_64 can go to 16TiB.

  Or virtually any Suse, Debian, or other distro that isn't as confined
in choice as RHEL.

>>   but I've yet to meet 
>> people who are very happy with the results over a TB, or 2.
> 
> I'm one. Depending on the hardware ext3 can be as good as 10-15% slower than 
> xfs on such a size. 

  More like 25-40% on RHEL 4.  RHEL 5 performs a lot better in most use
cases. Generally to making ext3 perform well requires writeback
journaling which sort of throws out most of the argument for ext3.

> And that is, IMHO, not worth the xfs hassle (manual 
> intervention on kernel updates, dangerous full fillsystem situations, less 
> mainstream...).

  If you run centos you just point your yum config at the centosplus
repo, and be sure you are running the centosplus kernel.  Ext3 people
wave their hands and talk about the dangers of xfs, and reiserfs, but
I've used them for years in production, and never had an issue.  Now yes
if you are using a laptop, or crappy ide/sata drives.  Ext3 is a safer
choice.  When your drive goes horribly wrong ext3 is going to gives you
the best chance of recovery.  In general if you're running a 10TB
filesystem you're on a solid raid array, and run regular backups.  (If
you have 10TB of data and don't have backups you are just plain insane.)