Copying a large file system (again)

[Tino Schwarze]

Hi Brice,

On Wed, Dec 05, 2007 at 08:31:55PM +0100, Brice Figureau wrote:

> >> > > I need to copy the whole /backup/backuppc directory structure in one
> >> > > pass so that hardlinks are preserved. Or I'd need some tool
> >> > > which could
> >> > > preserve the hardlinks another way.
> >> >
> >> > http://backuppc.sourceforge.net/faq/BackupPC.html#other_installation_topics
> >> >
> >> > The section on Copying the Pool may help, specifically using
> >> > BackupPC_tarPCCopy.  If that does not work, you may be able to copy
> >> the
> >> > pool over first, then do the hosts one at a time and try to manually
> >> run
> >> > BackupPC_link.
> >>
> >> Oh well, that should do the trick. I'll try copying the pool first, then
> >> use BackupPC_tarPCCopy to get a tar with almost only hardlinks in it,
> >> pointing to the pool I've already copied.
> >
> > I successfully copied the pool! :-) It took amost two days. Now I'm
> > running BackupPC_tarPCCopy and watching the machines... it seems there's
> > still some glitch, this time with the destination machine (PE1800 with
> > 3x750 GB SATA PERC RAID). I observe the following:
> >
> > Everything runs fine, both machines have I/O (I watch using vmstat), the
> > receiving tar tells me about the files it unpacks (mostly hardlinks to
> > the pool). After some time, the destination machine slows down
> > considerably, doing almost no I/O any more with 25%-50% I/O wait (it's
> > got 4 cores, source has only one and a lot of I/O wait). Using netstat I
> > see that the receive buffer is quite full (about a meg), the source
> > machine stops sending data and is 100% idle.
> >
> > Here is some "vmstat 1" output of the destination machine when it's
> > getting slow:

[...]

> I think that your system is starting writing dirty pages cached (see the
> 1.2GB of cache) to disk.
> Is it an ext3 filesystem (I mean the destination)?

It's an xfs with noatime option.

> To know if that's the write-back you can cat /proc/meminfo and see if
> writeback is > 0.

Writeback is always 0, dirty gets some kB from time to time.

> The following might increase your thoughput:
>  - echo "1" > /proc/sys/vm/dirty_background_ratio
>  - echo "2" > /proc/sys/vm/dirty_ratio
> This will force the dirty writeback to start early and continuously.

I've got /proc/sys/vm/dirty_background_ratio set to 1 and dirty_ration
set to 5 now. This seems to have improved the situation.

I've got the following command running to see which processes are
waiting for disk:
  while sleep 1 ; do ps fax | grep 'D[+<]\|D.*[p]dflush' ; done
and pdflush is showing up more regularly, but the system's
responsiveness doesn't suffer. :-)

Thanks,

Tino.

-- 
www.craniosacralzentrum.de
www.lebensraum11.de
www.spiritualdesign-chemnitz.de

Tino Schwarze * Parkstraße 17h * 09120 Chemnitz