MIME-Version: 1.0
References: <s0491qrn-343s-0757-8sn5-120rr8610qqq@tzk.arg>
In-Reply-To: <s0491qrn-343s-0757-8sn5-120rr8610qqq@tzk.arg>
From: Ron Johnson <ronljohnsonjr@gmail.com>
Date: Tue, 26 Aug 2025 18:00:09 -0400
Message-ID: <CANzqJaDGUPKM2h74wFb2DOgbKPnAe+c352eMTvehQ8D+vfxidA@mail.gmail.com>
Subject: Re: In-order pg_dump (or in-order COPY TO)
To: pgsql-general <pgsql-general@postgresql.org>
Content-Type: multipart/alternative; boundary="0000000000005d62d8063d4bcc14"
Archived-At: <https://www.postgresql.org/message-id/CANzqJaDGUPKM2h74wFb2DOgbKPnAe%2Bc352eMTvehQ8D%2BvfxidA%40mail.gmail.com>
Precedence: bulk

--0000000000005d62d8063d4bcc14
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Aug 26, 2025 at 3:44=E2=80=AFPM Dimitrios Apostolou <jimis@gmx.net>=
 wrote:

> Hello list,
>
> I am storing dumps of a database (pg_dump custom format) in a
> de-duplicating backup server. Each dump is many terabytes in size, so
> deduplication is very important. And de-duplication itself is based on
> rolling checksums which is pretty flexible, it can compensate for blocks
> moving by some offset.
>

This might be a silly question, but why are you using -Fc to
create multi-TB dumps instead of -Fd?  If nothing else, the parallelization
granted by -Fd will greatly speed up the dumps.

--=20
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

--0000000000005d62d8063d4bcc14
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Tue, Aug 26, 2025 at 3:44=E2=80=AFPM D=
imitrios Apostolou &lt;<a href=3D"mailto:jimis@gmx.net">jimis@gmx.net</a>&g=
t; wrote:</div><div class=3D"gmail_quote gmail_quote_container"><blockquote=
 class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px so=
lid rgb(204,204,204);padding-left:1ex">Hello list,<br>
<br>
I am storing dumps of a database (pg_dump custom format) in a <br>
de-duplicating backup server. Each dump is many terabytes in size, so <br>
deduplication is very important. And de-duplication itself is based on <br>
rolling checksums which is pretty flexible, it can compensate for blocks <b=
r>
moving by some offset.<br>
</blockquote></div><div><br clear=3D"all"></div><div>This might be a silly =
question, but why are you using -Fc to create=C2=A0multi-TB dumps instead o=
f -Fd?=C2=A0 If nothing else, the parallelization granted by -Fd will great=
ly speed up the dumps.</div><div><br></div><span class=3D"gmail_signature_p=
refix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D=
"ltr">Death to &lt;Redacted&gt;, and butter sauce.<div>Don&#39;t boil me, I=
&#39;m still alive.<br><div><div>&lt;Redacted&gt; lobster!</div></div></div=
></div></div></div>

--0000000000005d62d8063d4bcc14--