Subject: Re: Purpose of pg_dump tar archive format?

From: Ron Johnson <[email-masked]>
Date: Tue, 4 Jun 2024 15:15:06 -0400

On Tue, Jun 4, 2024 at 2:55 PM Rob Sargent <[email-masked]> wrote:

>
>
> On 6/4/24 11:40, Shaheed Haque wrote:
> >
> > We use it. I bet lots of others do too.
> >
> >
>
> Of course.  There are lots of small, real, useful databases in the wild.
>

But why tar instead of custom? That was part of my original question.

---

From: Rob Sargent <[email-masked]>
Date: Tue, 4 Jun 2024 13:37:14 -0600

> But why tar instead of custom? That was part of my original question.
That I can't say.  Familiarity?  Fewer keystrokes?

---

From: Gavin Roy <[email-masked]>
Date: Tue, 4 Jun 2024 15:47:22 -0400

> But why tar instead of custom? That was part of my original question.

I've found it pretty useful for programmatically accessing data in a dump
for large databases outside of the normal pg_dump/pg_restore workflow. You
don't have to seek through one large binary file to get to the data section
to get at the data.

---

From: Shaheed Haque <[email-masked]>
Date: Tue, 4 Jun 2024 22:35:21 +0100

> don't have to seek through one large binary file to get to the data section

This is true for us too; specifically, tar, including with compression, is
very convenient for both CLI and Python ecosystems.

---

From: Ron Johnson <[email-masked]>
Date: Tue, 4 Jun 2024 19:36:34 -0400

> don't have to seek through one large binary file to get to the data section

Interesting.  Please explain, though, since a big tarball _is_ "one large
binary file" that you have to sequentially scan.  (I don't know the
internal structure of custom format files, and whether they have file
pointers to each table.)

Is it because you need individual .dat "COPY" files for something other
than loading into PG tables (since pg_restore --table=xxxx does that, too),
and directory format archives can be inconvenient?

---

From: Gavin Roy <[email-masked]>
Date: Wed, 5 Jun 2024 10:22:35 -0400

> internal structure of custom format files, and whether they have file

Not if you untar it first.

> and directory format archives can be inconvenient?

In the past I've used it for data analysis outside of Postgres.
-- 
*Gavin M. Roy*
CTO
AWeber