From: Scott Ribe <scott_ribe@elevated-dev.com>
To: Cédric Villemain <cedric.villemain.debian@gmail.com>
Cc: [ADMIN] <pgsql-admin@postgresql.org>
Subject: Re: rsync and streaming replication
Date: Tue, 15 Nov 2011 08:59:10 -0700
Message-ID: <640CEEBE-CB12-4FC9-9758-0319A49E7ACF@elevated-dev.com> (raw)
In-Reply-To: <CAF6yO=2Cm_1sq7PZ=xRp5t+Oxu3e+bFWiRX+V2TOce4b5kauKg@mail.gmail.com>
References: <CALnckSpEi-6JjfEFLUcp13nScHbbcywFcmhn51iK2sj_eXbEkQ@mail.gmail.com>
<87pqgxf8p9.fsf@comcast.net>
<CALnckSqDpZi=_w8XckMLBHu_V4kgtn+EqNsqFcNgLtbcYorzDA@mail.gmail.com>
<CAF6yO=0UEB82UP2Lusd2xfrY+7TyVpn9pcFdDv1=_w9Rf=FqXg@mail.gmail.com>
<CALnckSqdG0aQ09nUhUeGWZ8SC8J25WEHHe4wD0jnz7K2+Kov5A@mail.gmail.com>
<639B3C52-BF6D-4CE9-A710-75D369E9607E@elevated-dev.com>
<CAF6yO=3WQMsa5WQkHNbfW9URnto5M94U5U9ZEQ3LS=dLRri2dw@mail.gmail.com>
<39A965E0-DE37-41F3-8D69-321419BB8B09@elevated-dev.com>
<CAF6yO=2Cm_1sq7PZ=xRp5t+Oxu3e+bFWiRX+V2TOce4b5kauKg@mail.gmail.com>
On Nov 15, 2011, at 8:30 AM, Cédric Villemain wrote:
> Seriously, I did. Is my post "just for the value : rsync --checksum is
> the option to use to prevent copying of **identical files**" incorrect
> ?
It's at least incomplete and somewhat misleading. But I guess you could say the same about my post; we seem to be focusing on 2 different aspects of its behavior ;-)
> OP contains "It looks that all database files do not have the same
> modification date in the master node and in the slave nodes, so the
> rsync copies quite all the database from the new master to the
> slaves."
Yes, and rsync should only be copying changed blocks in that case, of which you are aware, but which OP did not seem to realize.
> One benefit is when files are in fact identical on both side, so that
> rsync does not have to process checksum for each blocks on source and
> destination. (when there are few changes, we expect rsync to copy only
> those few changes, with or without --checksum).
Well, but it does calculate checksums on the entire contents of both files (which takes as much I/O and about as much CPU as calculating checksums for each block), even when timestamps & sizes are identical.
For the OP's case, identical files with differing timestamps, the only potential savings is from not exchanging checksums over the network, which is not likely to offer any meaningful improvement in performance, which still leaves open the question as to why rsync is so slow in that, when we know it is usually relatively fast to sync two servers with few differences.
Would be nice to actually hear from OP regarding file sizes/counts & network bandwidth & disks & and so on ;-)
--
Scott Ribe
scott_ribe@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: scott_ribe@elevated-dev.com, cedric.villemain.debian@gmail.com
Subject: Re: rsync and streaming replication
In-Reply-To: <640CEEBE-CB12-4FC9-9758-0319A49E7ACF@elevated-dev.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox