Received: from maia.hub.org (maia-2.hub.org [200.46.204.251]) by mail.postgresql.org (Postfix) with ESMTP id 1CCEA609F22 for ; Wed, 16 Nov 2011 13:23:53 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.251]) (amavisd-maia, port 10024) with ESMTP id 20536-07 for ; Wed, 16 Nov 2011 17:23:46 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0-rc2 Received: from makus.postgresql.org (makus.postgresql.org [98.129.198.125]) by mail.postgresql.org (Postfix) with ESMTP id 6BE05609F21 for ; Wed, 16 Nov 2011 13:23:46 -0400 (AST) Received: from asbnvacz-mailrelay01.megapath.net ([207.145.128.243]) by makus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1RQjD7-0005yx-8X for pgsql-admin@postgresql.org; Wed, 16 Nov 2011 17:23:45 +0000 Received: from mail8.sea5.speakeasy.net (mail8.sea5.speakeasy.net [69.17.117.53]) by asbnvacz-mailrelay01.megapath.net (Postfix) with ESMTP id 74AB8A7158F for ; Wed, 16 Nov 2011 12:23:32 -0500 (EST) Received: (qmail 11909 invoked from network); 16 Nov 2011 17:23:31 -0000 Received: by simscan 1.4.0 ppid: 9737, pid: 30610, t: 1.4299s scanners: clamav: 0.88.2/m:52/d:13495 spam: 3.0.4 Received: from 25-76-42-72.skybeam.com (HELO [192.168.2.16]) (sribe@elevated-dev.com@[72.42.76.25]) (envelope-sender ) by mail8.sea5.speakeasy.net (qmail-ldap-1.03) with AES128-SHA encrypted SMTP for ; 16 Nov 2011 17:23:30 -0000 Subject: Re: rsync and streaming replication Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Scott Ribe In-Reply-To: Date: Wed, 16 Nov 2011 10:23:28 -0700 Cc: =?iso-8859-1?Q?C=E9dric_Villemain?= , "[ADMIN]" Content-Transfer-Encoding: quoted-printable Message-Id: <48B4CE29-7776-4D7A-8AA5-D45A4F77BB75@elevated-dev.com> References: <87pqgxf8p9.fsf@comcast.net> <639B3C52-BF6D-4CE9-A710-75D369E9607E@elevated-dev.com> <39A965E0-DE37-41F3-8D69-321419BB8B09@elevated-dev.com> <640CEEBE-CB12-4FC9-9758-0319A49E7ACF@elevated-dev.com> To: Jean-Armel Luce X-Mailer: Apple Mail (2.1084) X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.18 tagged_above=-10 required=5 tests=BAYES_00=-1.9, SARE_SPEC_REPLICA=0.72 X-Spam-Level: X-Archive-Number: 201111/90 X-Sequence-Number: 38684 On Nov 16, 2011, at 10:11 AM, Jean-Armel Luce wrote: > You are right. I used -a, and I was wanting to be more meaningful so I = wrote --all in my post. > Please read --archive insterad of --all Oh, OK. Still seems odd that it took so much longer. Granted, for the = files with different timestamps but identical contents, it then syncs = them. But it does so by checksumming blocks, comparing checksums, and = sending only blocks that are different over the network. Granted, it has = to send some checksums over the network, but that's pretty minor = traffic. I believe you said you'd seen 125Mb/s over the network? Is that = actually accurate? Does the network connection have high latency? Also, I believe you said -z seemed to slow it down? That has not been my = experience at all with rsync'ing pg databases. Between all the values = that are stored as plain text, and the redundancies in indexes, I = usually see a good speed increase from compressing the data in transit. I'm certainly glad that you've got a 3x speed increase--that's = significant progress. But still, something seems odd about the = performance you've reported, so I'm left wondering about what could = cause that. Disk performance glitch at one end or the other, network = performance, CPU load??? One thing worth doing I think is to use --stats on every test, so you = can see every time how many files and how much data is actually = transferred. Also, if you're sitting there watching, sometimes = --progress can be informative... --=20 Scott Ribe scott_ribe@elevated-dev.com http://www.elevated-dev.com/ (303) 722-0567 voice