16 TB Disk Arrays and ext3fs woes

[Patrick_Boyd at Dell.com]

>From the ext3 FAQ (Not positive that it's up to date, but this is what I
found):
>Q: What is the largest possible size of an ext3 filesystem and of files
on ext3?
>
>inspired by Andreas Dilger, suggested by Christian Kujau:
>
>Ext3 can support files up to 1TB. With a 2.4 kernel the filesystem size
is limited by the maximal block device >size, which is 2TB. In 2.6 the
maximum (32-bit CPU) limit is of block devices is 16TB, but ext3
supports only up to >4TB.

>From what I can dig up if your filesystem is using 4KB blocks then you
are limited to 16TB exactly. So if you are just slightly over 16TB that
could be your issue. To get larger sizes you need to go over a 4KB block
size to 8KB blocks. 



-----Original Message-----
From: linux-poweredge-bounces at dell.com
[mailto:linux-poweredge-bounces at dell.com] On Behalf Of Fischer, Carl
Sent: Monday, August 06, 2007 7:55 AM
To: linux-poweredge-Lists
Subject: 16 TB Disk Arrays and ext3fs woes

   I'm running a 16 TB disk array on a PE2950 with a PERC 5/E
(megaraid_sas kernel module 3.09-1).  I expanded a pre-existing 3.5 TB
partition to fill the volume (using LVM and resizefs).  Now, the volume
mounts just fine, but only root can write to it.  Everyone else gets a
"disk full" error.  I don't see any hardware errors in
/var/log/messages.  My suspicion is that the filesystem thinks that the
13.5 additional TB are reserved for root, but I tried adjusting that
parameter with tune2fs and had no luck.  

   Even with 64-bit CentOS installed, fsck.ext3 runs for a long time,
then seg-faults (only the seg-fault shows up in /var/log/messages).
I've compiled and installed the latest e2progs, with the same results.
Not sure if this is a known software issue or some kind of hardware
issue.  

Any suggestions would be appreciated.  I've read online that XFS might
work better for large disk arrays.  Does anyone have experience here?

Thanks,
  Carl

________________________________
Carl H. Fischer IV, PhD	 Phone:	 +1 (781) 981-6702 	
MIT Lincoln Laboratory S3-227	 Cell: 	+1 (339) 440-1849 	
244 Wood Street, Lexington, MA 02420	 Fax: 	+1 (781) 981-7271 	

_______________________________________________
Linux-PowerEdge mailing list
Linux-PowerEdge at dell.com
http://lists.us.dell.com/mailman/listinfo/linux-poweredge
Please read the FAQ at http://lists.us.dell.com/faq