Time-honoured problem : RHEL3 iowait performance

[Brendan Heading]

Robin Humble writes: 

> maybe.
> lvm sets a pathetically low read-ahead on the block device.
> what does this tell you for your raw and lvm block device? eg.
>   blockdev --getra /dev/vol_md0/1
>   blockdev --getra /dev/md0 
> 
> when I set the read-ahead on the lvm to the same value as the non-lvm it
> goes at very close to the same speed. I find the below to be a good
> (large file) value for both:
>   blockdev --setra 16384 <block dev>

Thanks Robin. LVM actually reports a readahead of 1024, where my raw disks 
are set to 120. 

I have vm-readahead set to min 1024 max 2048 in the sysctl... 

> tested on software raid0 and raid5. on a raw scsi disk lvm isn't
> noticably slower.
> I haven't tested the setra theory with hardware raid.

I will try this .. 

I have an old 1450 here with a raw SCSI disk included. In my benchmarks LVM 
always shows up slightly slower; that isn't unreasonable considering that it 
adds an extra layer of stuff between the syscall interface and the hardware. 
The "fun" associated with RAID5's characteristics may effect LVM in strange 
ways though .. 

Brendan