Higher writes on XFS filesystem via LVM than expected

Bond Masuda bond.masuda at jlbond.com
Wed Sep 1 08:38:55 CDT 2010


Hi Florian,

Does iotop not show any other process or processes being responsible for
the 110Mbit/sec writes?

I would almost like to see an iostat and iotop output of the same period
to compare... something like:

iotop -n 10 -b -d 5

and 

iostat -m 5 10

run at the same time.

Those outputs are normally consistent.... but it sounds like your
situation it is not?

Keep in mind, iostat/iotop show actual writes to disk. For example, XFS
does a lot of buffering, so one may write 500MBytes, but see the
iostat/iotop activity a few seconds later.

-Bond

On Wed, 2010-09-01 at 15:17 +0200, Florian Jauernig wrote:
> Hi List,
> 
> hopefully this list is a correct place to ask...
> 
> I am facing a strange problem with my NFS server: although network interface 
> statistics only show up about 21Mbit/s incoming traffic, I am observing 
> about 110 Mbit/s writes to my storage with iostat.
> 
> With iotop I can see, that the nfsd processes only use some thousand kbit/s. 
> But total disk writes per second are about 110Mbit/s reported by iotop are 
> again about 110 Mbit/s.
> 
> I have to mention that I am no storage expert, but I tried my best with 
> searching for best practises in the internet of how to configure my setup.
> 
> My configuration:
> 
> We have a good old Poweredge 1850 acting as a NFS server for 3 VMware ESX 
> servers (Poweredge R510).
> 
> As storage I have an EMC CX300 with 2 LUN's. Each of the LUN's is a RAID 10 
> of 10 disks. The storage is directly connected (no switch, etc). Each LUN is 
> owned by another Storage Processor. We have multipathd running so in case a 
> fibre link is going down we have failover.
> 
> I created one big volume with LVM that consists of these 2 LUN's.
> 
> As filesystem I am using XFS.
> 
> I tried my best so everything is aligned (Partitions accordingly to the 
> stripe size of the LUN's), Metadata of the LVM PV's is hopefully also 
> aligned.
> 
> XFS is hopefully correctly tuned with sunit and swidth, and mounted with 
> relatime and nobarrier.
> 
> On kernel level I am using noop scheduler (so the CX300 storage processors 
> can do the scheduling work and do not interfere with each other).
> 
> Has anyone an idea what might cause the much higher constant writes to disk 
> than expected by NFS? If you need further information like exact 
> configuration or logfiles I will append those.
> 
> Any help is greatly appreciated!
> 
> Thanks in advance
> 
> Florian
> _______________________________________________
> Linux-PowerEdge mailing list
> Linux-PowerEdge at dell.com
> https://lists.us.dell.com/mailman/listinfo/linux-poweredge
> Please read the FAQ at http://lists.us.dell.com/faq




More information about the Linux-PowerEdge mailing list