Subject: Re: rsync and streaming replication
Mime-Version: 1.0 (Apple Message framework v1084)
Content-Type: text/plain; charset=iso-8859-1
From: Scott Ribe <scott_ribe@elevated-dev.com>
In-Reply-To: 
 <CAF6yO=2Cm_1sq7PZ=xRp5t+Oxu3e+bFWiRX+V2TOce4b5kauKg@mail.gmail.com>
Date: Tue, 15 Nov 2011 08:59:10 -0700
Cc: "[ADMIN]" <pgsql-admin@postgresql.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <640CEEBE-CB12-4FC9-9758-0319A49E7ACF@elevated-dev.com>
References: 
 <CALnckSpEi-6JjfEFLUcp13nScHbbcywFcmhn51iK2sj_eXbEkQ@mail.gmail.com>
	<87pqgxf8p9.fsf@comcast.net>
	<CALnckSqDpZi=_w8XckMLBHu_V4kgtn+EqNsqFcNgLtbcYorzDA@mail.gmail.com>
	<CAF6yO=0UEB82UP2Lusd2xfrY+7TyVpn9pcFdDv1=_w9Rf=FqXg@mail.gmail.com>
	<CALnckSqdG0aQ09nUhUeGWZ8SC8J25WEHHe4wD0jnz7K2+Kov5A@mail.gmail.com>
	<639B3C52-BF6D-4CE9-A710-75D369E9607E@elevated-dev.com>
	<CAF6yO=3WQMsa5WQkHNbfW9URnto5M94U5U9ZEQ3LS=dLRri2dw@mail.gmail.com>
	<39A965E0-DE37-41F3-8D69-321419BB8B09@elevated-dev.com>
	<CAF6yO=2Cm_1sq7PZ=xRp5t+Oxu3e+bFWiRX+V2TOce4b5kauKg@mail.gmail.com>
To: =?iso-8859-1?Q?C=E9dric_Villemain?= <cedric.villemain.debian@gmail.com>

On Nov 15, 2011, at 8:30 AM, C=E9dric Villemain wrote:

> Seriously, I did. Is my post "just for the value : rsync --checksum is
> the option to use to prevent copying of **identical files**" incorrect
> ?

It's at least incomplete and somewhat misleading. But I guess you could =
say the same about my post; we seem to be focusing on 2 different =
aspects of its behavior ;-)

> OP contains "It looks that all database files do not have the same
> modification date in the master node and in the slave nodes, so the
> rsync copies quite all the database from the new master to the
> slaves."

Yes, and rsync should only be copying changed blocks in that case, of =
which you are aware, but which OP did not seem to realize.

> One benefit is when files are in fact identical on both side, so that
> rsync does not have to process checksum for each blocks on source and
> destination. (when there are few changes, we expect rsync to copy only
> those few changes, with or without --checksum).

Well, but it does calculate checksums on the entire contents of both =
files (which takes as much I/O and about as much CPU as calculating =
checksums for each block), even when timestamps & sizes are identical.

For the OP's case, identical files with differing timestamps, the only =
potential savings is from not exchanging checksums over the network, =
which is not likely to offer any meaningful improvement in performance, =
which still leaves open the question as to why rsync is so slow in that, =
when we know it is usually relatively fast to sync two servers with few =
differences.

Would be nice to actually hear from OP regarding file sizes/counts & =
network bandwidth & disks & and so on ;-)

--=20
Scott Ribe
scott_ribe@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice