public inbox for [email protected]
help / color / mirror / Atom feedRe: Purpose of pg_dump tar archive format?
6+ messages / 4 participants
[nested] [flat]
* Re: Purpose of pg_dump tar archive format?
@ 2024-06-04 19:15 Ron Johnson <[email protected]>
2024-06-04 19:37 ` Re: Purpose of pg_dump tar archive format? Rob Sargent <[email protected]>
2024-06-04 19:47 ` Re: Purpose of pg_dump tar archive format? Gavin Roy <[email protected]>
0 siblings, 2 replies; 6+ messages in thread
From: Ron Johnson @ 2024-06-04 19:15 UTC (permalink / raw)
To: pgsql-generallists.postgresql.org <[email protected]>
On Tue, Jun 4, 2024 at 2:55 PM Rob Sargent <[email protected]> wrote:
>
>
> On 6/4/24 11:40, Shaheed Haque wrote:
> >
> > We use it. I bet lots of others do too.
> >
> >
>
> Of course. There are lots of small, real, useful databases in the wild.
>
But why tar instead of custom? That was part of my original question.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format?
2024-06-04 19:15 Re: Purpose of pg_dump tar archive format? Ron Johnson <[email protected]>
@ 2024-06-04 19:37 ` Rob Sargent <[email protected]>
1 sibling, 0 replies; 6+ messages in thread
From: Rob Sargent @ 2024-06-04 19:37 UTC (permalink / raw)
To: [email protected]
On 6/4/24 13:15, Ron Johnson wrote:
> On Tue, Jun 4, 2024 at 2:55 PM Rob Sargent <[email protected]> wrote:
>
>
>
> On 6/4/24 11:40, Shaheed Haque wrote:
> >
> > We use it. I bet lots of others do too.
> >
> >
>
> Of course. There are lots of small, real, useful databases in the
> wild.
>
> But why tar instead of custom? That was part of my original question.
>
That I can't say. Familiarity? Fewer keystrokes?
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format?
2024-06-04 19:15 Re: Purpose of pg_dump tar archive format? Ron Johnson <[email protected]>
@ 2024-06-04 19:47 ` Gavin Roy <[email protected]>
2024-06-04 21:35 ` Re: Purpose of pg_dump tar archive format? Shaheed Haque <[email protected]>
2024-06-04 23:36 ` Re: Purpose of pg_dump tar archive format? Ron Johnson <[email protected]>
1 sibling, 2 replies; 6+ messages in thread
From: Gavin Roy @ 2024-06-04 19:47 UTC (permalink / raw)
To: Ron Johnson <[email protected]>; +Cc: pgsql-generallists.postgresql.org <[email protected]>
On Tue, Jun 4, 2024 at 3:15 PM Ron Johnson <[email protected]> wrote:
>
> But why tar instead of custom? That was part of my original question.
>
I've found it pretty useful for programmatically accessing data in a dump
for large databases outside of the normal pg_dump/pg_restore workflow. You
don't have to seek through one large binary file to get to the data section
to get at the data.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format?
2024-06-04 19:15 Re: Purpose of pg_dump tar archive format? Ron Johnson <[email protected]>
2024-06-04 19:47 ` Re: Purpose of pg_dump tar archive format? Gavin Roy <[email protected]>
@ 2024-06-04 21:35 ` Shaheed Haque <[email protected]>
1 sibling, 0 replies; 6+ messages in thread
From: Shaheed Haque @ 2024-06-04 21:35 UTC (permalink / raw)
To: Gavin Roy <[email protected]>; +Cc: Ron Johnson <[email protected]>; pgsql-generallists.postgresql.org <[email protected]>
On Tue, 4 Jun 2024 at 20:47, Gavin Roy <[email protected]> wrote:
>
> On Tue, Jun 4, 2024 at 3:15 PM Ron Johnson <[email protected]>
> wrote:
>
>>
>> But why tar instead of custom? That was part of my original question.
>>
>
> I've found it pretty useful for programmatically accessing data in a dump
> for large databases outside of the normal pg_dump/pg_restore workflow. You
> don't have to seek through one large binary file to get to the data section
> to get at the data.
>
This is true for us too; specifically, tar, including with compression, is
very convenient for both CLI and Python ecosystems.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format?
2024-06-04 19:15 Re: Purpose of pg_dump tar archive format? Ron Johnson <[email protected]>
2024-06-04 19:47 ` Re: Purpose of pg_dump tar archive format? Gavin Roy <[email protected]>
@ 2024-06-04 23:36 ` Ron Johnson <[email protected]>
2024-06-05 14:22 ` Re: Purpose of pg_dump tar archive format? Gavin Roy <[email protected]>
1 sibling, 1 reply; 6+ messages in thread
From: Ron Johnson @ 2024-06-04 23:36 UTC (permalink / raw)
To: pgsql-general
On Tue, Jun 4, 2024 at 3:47 PM Gavin Roy <[email protected]> wrote:
>
> On Tue, Jun 4, 2024 at 3:15 PM Ron Johnson <[email protected]>
> wrote:
>
>>
>> But why tar instead of custom? That was part of my original question.
>>
>
> I've found it pretty useful for programmatically accessing data in a dump
> for large databases outside of the normal pg_dump/pg_restore workflow. You
> don't have to seek through one large binary file to get to the data section
> to get at the data.
>
Interesting. Please explain, though, since a big tarball _is_ "one large
binary file" that you have to sequentially scan. (I don't know the
internal structure of custom format files, and whether they have file
pointers to each table.)
Is it because you need individual .dat "COPY" files for something other
than loading into PG tables (since pg_restore --table=xxxx does that, too),
and directory format archives can be inconvenient?
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format?
2024-06-04 19:15 Re: Purpose of pg_dump tar archive format? Ron Johnson <[email protected]>
2024-06-04 19:47 ` Re: Purpose of pg_dump tar archive format? Gavin Roy <[email protected]>
2024-06-04 23:36 ` Re: Purpose of pg_dump tar archive format? Ron Johnson <[email protected]>
@ 2024-06-05 14:22 ` Gavin Roy <[email protected]>
0 siblings, 0 replies; 6+ messages in thread
From: Gavin Roy @ 2024-06-05 14:22 UTC (permalink / raw)
To: Ron Johnson <[email protected]>; +Cc: pgsql-general
On Tue, Jun 4, 2024 at 7:36 PM Ron Johnson <[email protected]> wrote:
> On Tue, Jun 4, 2024 at 3:47 PM Gavin Roy <[email protected]> wrote:
>
>>
>> On Tue, Jun 4, 2024 at 3:15 PM Ron Johnson <[email protected]>
>> wrote:
>>
>>>
>>> But why tar instead of custom? That was part of my original question.
>>>
>>
>> I've found it pretty useful for programmatically accessing data in a dump
>> for large databases outside of the normal pg_dump/pg_restore workflow. You
>> don't have to seek through one large binary file to get to the data section
>> to get at the data.
>>
>
> Interesting. Please explain, though, since a big tarball _is_ "one large
> binary file" that you have to sequentially scan. (I don't know the
> internal structure of custom format files, and whether they have file
> pointers to each table.)
>
Not if you untar it first.
> Is it because you need individual .dat "COPY" files for something other
> than loading into PG tables (since pg_restore --table=xxxx does that, too),
> and directory format archives can be inconvenient?
>
In the past I've used it for data analysis outside of Postgres.
--
*Gavin M. Roy*
CTO
AWeber
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2024-06-05 14:22 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-06-04 19:15 Re: Purpose of pg_dump tar archive format? Ron Johnson <[email protected]>
2024-06-04 19:37 ` Rob Sargent <[email protected]>
2024-06-04 19:47 ` Gavin Roy <[email protected]>
2024-06-04 21:35 ` Shaheed Haque <[email protected]>
2024-06-04 23:36 ` Ron Johnson <[email protected]>
2024-06-05 14:22 ` Gavin Roy <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox