Subject: Re: In-order pg_dump (or in-order COPY TO)

From: Tom Lane <[email-masked]>
Date: Wed, 27 Aug 2025 10:16:33 -0400

Dimitrios Apostolou <[email-masked]> writes:
> Dump is from PostgreSQL 16, it's pg_dump writing to stdout:

> pg_dump -v --format=custom --compress=none --no-toast-compression --serializable-deferrable db_name  |  borg create ...

Don't use --format=custom (and not -v either).  That causes pg_dump to
include the OIDs and pg_dump object IDs of all the tables and other
objects, which will all be different in a dump from the new server.
The actual data contents of the tables should be the same, but
apparently the differences in the entry headers are enough to
mislead borgbackup.

You might be well advised to manually examine the data you are
stuffing into borgbackup.  Right now we seem to be operating on
hypotheses, not facts, about what that looks like and how it's
different between your old and new server.

			regards, tom lane

---

From: Ron Johnson <[email-masked]>
Date: Wed, 27 Aug 2025 10:31:24 -0400

On Wed, Aug 27, 2025 at 10:16 AM Tom Lane <[email-masked]> wrote:

> > pg_dump -v --format=custom --compress=none --no-toast-compression
> --serializable-deferrable db_name  |  borg create ...
>
> objects,

That's interesting.  Why?  (Since isn't it supposed to be Bad to rely on
OIDs?)

-- 
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

---

From: Tom Lane <[email-masked]>
Date: Wed, 27 Aug 2025 10:42:57 -0400

> That's interesting.  Why?  (Since isn't it supposed to be Bad to rely on

-v in a text-format dump includes that data for debugging purposes:

--
-- TOC entry 1401 (class 1255 OID 16499)
-- Name: fipshash(text); Type: FUNCTION; Schema: public; Owner: postgres
--

(The "TOC entry" comment line wouldn't be there without -v.)
Then custom format has to store the same info so that pg_restore
can produce this identical text output on demand.

Yeah, this is all pretty historical, but nobody wants to change it
at this point.

			regards, tom lane

---

From: Ron Johnson <[email-masked]>
Date: Wed, 27 Aug 2025 11:12:50 -0400

> -v in a text-format dump includes that data for debugging purposes:

Ah, so the culprit is "-v".  I like using -v, redirecting it to a log file
(more info is almost always better), but then I rarely use pg_dump, and
never pipe it to de-duplicators.  (ExaGrid is supposed to deduplicate, but
that's not going to stop me from using pgbackrest, compression and
encryption; PCI auditors care about that, not deduplication.)