Feasability of VERY large ext3 file system?

Hubbard, David dhubbard at dino.hostasaurus.com
Wed Dec 4 06:41:00 CST 2002


From: Basil Hussain [mailto:basil.hussain at kodakweddings.com] 
> 
> Hi all,
> 
> I am soon considering setting up an external RAID array to be 
> attached to one of my PowerEdge servers. The RAID array is an
> external one with it's own hardware RAID controller, attached
> via U160 SCSI. It is likely that this array will be at least
> 1.4 terabytes, possibly up to 2.2 terabytes. I plan
> to run Redhat 8.0 on this server.
> 
> I have a few questions that I'm hoping someone might have 
> some answers for:
> 
> * Would a single multi-TB file system using ext3 be pushing 
> things a bit?

Don't have exactly the same problem but do have a similar
issue:

I have what is supposed to be a 600 GB-usable disk provided
by a 3ware controller and (6) 120 GB IDE drives in a raid 5
config.  Some of the more interesting things are 2.4.18-10
reporting this on boot:

SCSI device sda: 1171864320 512-byte hdwr sectors (-499516 MB)

df shows:

/dev/sda1            572680832 122499596 421090736  23% /
/dev/sda3               505636      8308    471223   2% /tmp
/dev/sda2              3020172     65424   2801328   3% /var

I have a 512 MB swap volume so I have not yet figured out
where my other 37671288 kb (35 GB) out of a total of what
should be 614400000 kb has gone.  I just did this last week
and have opened a ticket with 3ware on it.  Their controller
shows what it reports to be a 600 GB drive, so I was wanting
my 600 GB from linux too.

If this is all ext3 that would explain it, but I hadn't
even thought of that...

> * The data stored would be organised in a directory hierarchy 
> only one level deep. How would ext3 cope with, say, 4000-5000
> directories off the file system's root?

Mine's acting as a disk-based backup server as an intermediary
to tape storage, so it just has quite a few 2 GB files that
NetBackup writes out as it backs servers up.  Seems to handle
it pretty well, the 3ware gets a nice 50 MB/sec writing speed
even in raid 5, 130 MB/sec reading.

> * How much space is lost due to journaling? Basically, given 
> a gigabyte of disk, what is the available formatted space?
> 
> If anyone could offer some advice, anecdotes, etc. on running 
> *large* file systems using ext3, I would be most grateful.

If you hear anything let me know, I'm interested too as I have
six more available channels on this 3ware and was thinking
about growing it to 1.2 TB once I figure out where my missing
37 GB has gone.  It's nice being able to add 600 GB for $1000
using IDE drives. :-)

David




More information about the Linux-PowerEdge mailing list