public inbox for [email protected]help / color / mirror / Atom feed
Re: Purpose of pg_dump tar archive format? 6+ messages / 4 participants [nested] [flat]
* Re: Purpose of pg_dump tar archive format? @ 2024-06-04 19:15 Ron Johnson <[email protected]> 0 siblings, 2 replies; 6+ messages in thread From: Ron Johnson @ 2024-06-04 19:15 UTC (permalink / raw) To: pgsql-generallists.postgresql.org <[email protected]> On Tue, Jun 4, 2024 at 2:55 PM Rob Sargent <[email protected]> wrote: > > > On 6/4/24 11:40, Shaheed Haque wrote: > > > > We use it. I bet lots of others do too. > > > > > > Of course. There are lots of small, real, useful databases in the wild. > But why tar instead of custom? That was part of my original question. ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format? @ 2024-06-04 19:37 Rob Sargent <[email protected]> parent: Ron Johnson <[email protected]> 1 sibling, 0 replies; 6+ messages in thread From: Rob Sargent @ 2024-06-04 19:37 UTC (permalink / raw) To: [email protected] On 6/4/24 13:15, Ron Johnson wrote: > On Tue, Jun 4, 2024 at 2:55 PM Rob Sargent <[email protected]> wrote: > > > > On 6/4/24 11:40, Shaheed Haque wrote: > > > > We use it. I bet lots of others do too. > > > > > > Of course. There are lots of small, real, useful databases in the > wild. > > But why tar instead of custom? That was part of my original question. > That I can't say. Familiarity? Fewer keystrokes? ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format? @ 2024-06-04 19:47 Gavin Roy <[email protected]> parent: Ron Johnson <[email protected]> 1 sibling, 2 replies; 6+ messages in thread From: Gavin Roy @ 2024-06-04 19:47 UTC (permalink / raw) To: Ron Johnson <[email protected]>; +Cc: pgsql-generallists.postgresql.org <[email protected]> On Tue, Jun 4, 2024 at 3:15 PM Ron Johnson <[email protected]> wrote: > > But why tar instead of custom? That was part of my original question. > I've found it pretty useful for programmatically accessing data in a dump for large databases outside of the normal pg_dump/pg_restore workflow. You don't have to seek through one large binary file to get to the data section to get at the data. ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format? @ 2024-06-04 21:35 Shaheed Haque <[email protected]> parent: Gavin Roy <[email protected]> 1 sibling, 0 replies; 6+ messages in thread From: Shaheed Haque @ 2024-06-04 21:35 UTC (permalink / raw) To: Gavin Roy <[email protected]>; +Cc: Ron Johnson <[email protected]>; pgsql-generallists.postgresql.org <[email protected]> On Tue, 4 Jun 2024 at 20:47, Gavin Roy <[email protected]> wrote: > > On Tue, Jun 4, 2024 at 3:15 PM Ron Johnson <[email protected]> > wrote: > >> >> But why tar instead of custom? That was part of my original question. >> > > I've found it pretty useful for programmatically accessing data in a dump > for large databases outside of the normal pg_dump/pg_restore workflow. You > don't have to seek through one large binary file to get to the data section > to get at the data. > This is true for us too; specifically, tar, including with compression, is very convenient for both CLI and Python ecosystems. ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format? @ 2024-06-04 23:36 Ron Johnson <[email protected]> parent: Gavin Roy <[email protected]> 1 sibling, 1 reply; 6+ messages in thread From: Ron Johnson @ 2024-06-04 23:36 UTC (permalink / raw) To: pgsql-general On Tue, Jun 4, 2024 at 3:47 PM Gavin Roy <[email protected]> wrote: > > On Tue, Jun 4, 2024 at 3:15 PM Ron Johnson <[email protected]> > wrote: > >> >> But why tar instead of custom? That was part of my original question. >> > > I've found it pretty useful for programmatically accessing data in a dump > for large databases outside of the normal pg_dump/pg_restore workflow. You > don't have to seek through one large binary file to get to the data section > to get at the data. > Interesting. Please explain, though, since a big tarball _is_ "one large binary file" that you have to sequentially scan. (I don't know the internal structure of custom format files, and whether they have file pointers to each table.) Is it because you need individual .dat "COPY" files for something other than loading into PG tables (since pg_restore --table=xxxx does that, too), and directory format archives can be inconvenient? ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Purpose of pg_dump tar archive format? @ 2024-06-05 14:22 Gavin Roy <[email protected]> parent: Ron Johnson <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Gavin Roy @ 2024-06-05 14:22 UTC (permalink / raw) To: Ron Johnson <[email protected]>; +Cc: pgsql-general On Tue, Jun 4, 2024 at 7:36 PM Ron Johnson <[email protected]> wrote: > On Tue, Jun 4, 2024 at 3:47 PM Gavin Roy <[email protected]> wrote: > >> >> On Tue, Jun 4, 2024 at 3:15 PM Ron Johnson <[email protected]> >> wrote: >> >>> >>> But why tar instead of custom? That was part of my original question. >>> >> >> I've found it pretty useful for programmatically accessing data in a dump >> for large databases outside of the normal pg_dump/pg_restore workflow. You >> don't have to seek through one large binary file to get to the data section >> to get at the data. >> > > Interesting. Please explain, though, since a big tarball _is_ "one large > binary file" that you have to sequentially scan. (I don't know the > internal structure of custom format files, and whether they have file > pointers to each table.) > Not if you untar it first. > Is it because you need individual .dat "COPY" files for something other > than loading into PG tables (since pg_restore --table=xxxx does that, too), > and directory format archives can be inconvenient? > In the past I've used it for data analysis outside of Postgres. -- *Gavin M. Roy* CTO AWeber ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2024-06-05 14:22 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-06-04 19:15 Re: Purpose of pg_dump tar archive format? Ron Johnson <[email protected]> 2024-06-04 19:37 ` Rob Sargent <[email protected]> 2024-06-04 19:47 ` Gavin Roy <[email protected]> 2024-06-04 21:35 ` Shaheed Haque <[email protected]> 2024-06-04 23:36 ` Ron Johnson <[email protected]> 2024-06-05 14:22 ` Gavin Roy <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox