Feasability of VERY large ext3 file system?

Stephan Wonczak Wonczak at rrz.uni-koeln.de
Wed Dec 4 08:05:01 CST 2002

  Hi Basil!

On Wed, 4 Dec 2002, Basil Hussain wrote:

> Hi all,
> I am soon considering setting up an external RAID array to be attached to
> one of my PowerEdge servers. The RAID array is an external one with it's own
> hardware RAID controller, attached via U160 SCSI. It is likely that this
> array will be at least 1.4 terabytes, possibly up to 2.2 terabytes. I plan
> to run Redhat 8.0 on this server.

  You have to be careful with a RAID-array this large. The (newer) 2.4
kernels have a block device limit of 2TB, so 2.2TB all in one piece won't
work. RH AS 2.1 is even worse (this bit me just today); here the limit is
1 TB!

> I have a few questions that I'm hoping someone might have some answers for:
> * Would a single multi-TB file system using ext3 be pushing things a bit?

  That depends. If you have few large files, then no problem. If you have
lots and lots of small files (> 1000000) then you may run out of inodes,
even if there is lots of unallocated space left. (you can cater for this
during fs-creation time, though). SGI XFS on the other hand has no such
limit, and has better performance, too (additionally this is meant as a
journaling file system, not an ext2 with a journal tacked on)

> * The data stored would be organised in a directory hierarchy only one level
> deep. How would ext3 cope with, say, 4000-5000 directories off the file
> system's root?

  Um, no idea. I know that some versions of ext[23] suck royally when you
have a lot of files in a single directory... finding a single file can
take several seconds in extreme circumstances. I remember this being fixed
in 2.5 kernels, but I don't know wether this has been backported to 2.4

> * How much space is lost due to journaling? Basically, given a gigabyte of
> disk, what is the available formatted space?

  As others remarked, the journal size is independent (well nearly) of the
size of the file system.

> If anyone could offer some advice, anecdotes, etc. on running *large* file
> systems using ext3, I would be most grateful.

  As soon as my &^$&^ RAID box finishes reconfiguring, we will test a mail
server installation on a 500GB ext3 partition (with about 1.5 million
files, and some directories with >100k files). Let's hope it does not fall
over! (nb: We did these test with XFS, but RH AS does not support this

                Dipl. Chem. Dr. Stephan Wonczak
		Institut fuer Angewandte Informatik (ZAIK)
                Regionales Rechenzentrum der Universitaet zu Koeln (RRZK)
                Universitaet zu Koeln
                Robert-Koch-Strasse 10
                50931 Koeln
                Tel: ++49/(0)221/478-5577
		Fax: ++49/(0)221/478-5590

More information about the Linux-PowerEdge mailing list