MIME-Version: 1.0
References: <49351195.20220516095634@am-soft.de>
In-Reply-To: <49351195.20220516095634@am-soft.de>
From: Jeff Janes <jeff.janes@gmail.com>
Date: Wed, 18 May 2022 23:46:37 -0400
Message-ID: 
 <CAMkU=1xmXSp-enz-EhLi0jL0-NJPFNxx3CEx7UJOh=EAxdLSig@mail.gmail.com>
Subject: Re: How to get a more RSYNC compatible output of pg_dump?
To: =?UTF-8?Q?Thorsten_Sch=C3=B6ning?= <tschoening@am-soft.de>
Cc: Pgsql-admin <pgsql-admin@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="0000000000003ec7d705df553989"
Archived-At: 
 <https://www.postgresql.org/message-id/CAMkU%3D1xmXSp-enz-EhLi0jL0-NJPFNxx3CEx7UJOh%3DEAxdLSig%40mail.gmail.com>
Precedence: bulk

--0000000000003ec7d705df553989
Content-Type: text/plain; charset="UTF-8"

>
> Though, with the large dumps it seems to me that with every slight
> change in the actual data the entire dump gets downloaded again. I'm
> already using uncompressed dumps in the hope that the output is more
> stable and RSYNC better able to recognize unchanged parts. But I guess
> that most changes in the dumped data simply result in all subsequent
> data being that misplaced compared to what RSYNC reads against, that
> it's like downloading the whole file again in the end.
>

For me the rsync differential algorithm on dump files just works with no
special preparations.  I think you need to investigate this from the rsync
side, not PostgreSQL.  For example, if you change a dump file just by
adding or deleting one character near the beginning, how does rsync respond
to that?

Cheers,

Jeff

--0000000000003ec7d705df553989
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><div class=3D"gmail_quote"><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left=
:1px solid rgb(204,204,204);padding-left:1ex"><br>
Though, with the large dumps it seems to me that with every slight<br>
change in the actual data the entire dump gets downloaded again. I&#39;m<br=
>
already using uncompressed dumps in the hope that the output is more<br>
stable and RSYNC better able to recognize unchanged parts. But I guess<br>
that most changes in the dumped data simply result in all subsequent<br>
data being that misplaced compared to what RSYNC reads against, that<br>
it&#39;s like downloading the whole file again in the end.<br></blockquote>=
<div><br></div><div>For me the rsync differential algorithm on dump files j=
ust works with no special preparations.=C2=A0 I think you need to investiga=
te this from the rsync side, not PostgreSQL.=C2=A0 For example, if you chan=
ge a dump file just by adding or deleting one character near the beginning,=
 how does rsync respond to that?=C2=A0</div><div><br></div><div>Cheers,</di=
v><div><br></div><div>Jeff</div></div></div>

--0000000000003ec7d705df553989--