csync2 for very large file sets (was: Re: Redundant NFS storage setup)

[Tino Schwarze]

Hi Matthias et al,

On Wed, Nov 28, 2007 at 12:40:59PM +0100, Matthias Saou wrote:

> I have to admit that csync2 is really neat, and replaces in an elegant
> fashion some ugly rsync+ssh scripts I had been previously using ;-)

I've been hearing good things about csync2 on this list recently.
And I've been hearing good things about xfs too. My current situation is
as follows: We've got a SW-RAID5 with a 900 GB reiserfs file system on it.
The file system contains BackupPC data of several servers. BackupPC uses
a pool of files and hardlinks to save space on backups. That is, the
pool has about 5 million files and 20 million directories.

Since we've had problems with the SW RAID, the reiserfs suffered and I'm
about to --rebuild-tree the second time. A new machine (PE1800) is ready
to take all the data and I'm considering switching file systems since
I'm not feeling good about keeping this reiserfs any longer.

My question: Will csync2 be able to handle such an amount of files and
preserve hardlinks correctly? On another occasion I tried rsync'ing the
data but aborted after rsync took over 1 GB of RAM while building the
file list - the machine only has a Gig.

I'm assuming that xfs will cope well with the data.

Any advice how to cope with such an amount of data?

Thanks,

Tino.

-- 
www.craniosacralzentrum.de
www.lebensraum11.de
www.spiritualdesign-chemnitz.de

Tino Schwarze * Parkstraße 17h * 09120 Chemnitz