MIME-Version: 1.0
References: <CANzqJaAHOSNiOhe4D2hOBB46=puWQOZb=s=VQdNmUSOtOmRG5A@mail.gmail.com>
 <25e9749c-ff38-4832-9b26-386cac33b3d8@aklaver.com> <CANzqJaCMR-5eLMYy0ukywWeKpwySCAMQRrgKmW7A2CZ-mr9+0Q@mail.gmail.com>
 <7005ac8c-2f83-4122-9172-04bca268f987@aklaver.com> <CANzqJaBiUpeFExzS-wi8KzmZcxEEmi1FUVkoyL5=8FEx13S5dg@mail.gmail.com>
 <f3d35e91-9b80-4985-a1ca-fa0a5d8639d2@aklaver.com> <CAHAc2jcACDXv8=5EVO0YVVjANP=oj_=eeYeLXiG3LQC+kU0ewA@mail.gmail.com>
 <4b28e899-d802-43ce-b20b-655a9077f08f@gmail.com> <CANzqJaB86-8qozgb5Fmd48PJa4CrVo3o8BFGAiG0w6ynJfJJpA@mail.gmail.com>
 <CAFVAjJH3bA-rZNEyjDctD0TOZMjcyVqA7Ac_-xkBDaM8UUPFkA@mail.gmail.com> <CANzqJaDonyXAvPpNi5ZX3WUrCCxXfPHAzOKqab29Zq-WZ5UEaQ@mail.gmail.com>
In-Reply-To: <CANzqJaDonyXAvPpNi5ZX3WUrCCxXfPHAzOKqab29Zq-WZ5UEaQ@mail.gmail.com>
From: Gavin Roy <gavinr@aweber.com>
Date: Wed, 5 Jun 2024 10:22:35 -0400
Message-ID: <CAFVAjJGcpV1V9tD8r-9mNH64KcYaRQE2D1pTM0Lq8n-GDAWx0g@mail.gmail.com>
Subject: Re: Purpose of pg_dump tar archive format?
To: Ron Johnson <ronljohnsonjr@gmail.com>
Cc: pgsql-general <pgsql-general@postgresql.org>
Content-Type: multipart/alternative; boundary="000000000000def1b7061a254c24"
Archived-At: <https://www.postgresql.org/message-id/CAFVAjJGcpV1V9tD8r-9mNH64KcYaRQE2D1pTM0Lq8n-GDAWx0g%40mail.gmail.com>
Precedence: bulk

--000000000000def1b7061a254c24
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Jun 4, 2024 at 7:36=E2=80=AFPM Ron Johnson <ronljohnsonjr@gmail.com=
> wrote:

> On Tue, Jun 4, 2024 at 3:47=E2=80=AFPM Gavin Roy <gavinr@aweber.com> wrot=
e:
>
>>
>> On Tue, Jun 4, 2024 at 3:15=E2=80=AFPM Ron Johnson <ronljohnsonjr@gmail.=
com>
>> wrote:
>>
>>>
>>> But why tar instead of custom? That was part of my original question.
>>>
>>
>> I've found it pretty useful for programmatically accessing data in a dum=
p
>> for large databases outside of the normal pg_dump/pg_restore workflow. Y=
ou
>> don't have to seek through one large binary file to get to the data sect=
ion
>> to get at the data.
>>
>
> Interesting.  Please explain, though, since a big tarball _is_ "one large
> binary file" that you have to sequentially scan.  (I don't know the
> internal structure of custom format files, and whether they have file
> pointers to each table.)
>

Not if you untar it first.


> Is it because you need individual .dat "COPY" files for something other
> than loading into PG tables (since pg_restore --table=3Dxxxx does that, t=
oo),
> and directory format archives can be inconvenient?
>

In the past I've used it for data analysis outside of Postgres.
--=20
*Gavin M. Roy*
CTO
AWeber

--000000000000def1b7061a254c24
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"></div><br><div class=3D"gmail_quote"><div=
 dir=3D"ltr" class=3D"gmail_attr">On Tue, Jun 4, 2024 at 7:36=E2=80=AFPM Ro=
n Johnson &lt;<a href=3D"mailto:ronljohnsonjr@gmail.com" target=3D"_blank">=
ronljohnsonjr@gmail.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_=
quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,=
204);padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr">On Tue, Jun 4, 202=
4 at 3:47=E2=80=AFPM Gavin Roy &lt;<a href=3D"mailto:gavinr@aweber.com" tar=
get=3D"_blank">gavinr@aweber.com</a>&gt; wrote:<br></div><div class=3D"gmai=
l_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e=
x;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"=
><div dir=3D"ltr"></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cla=
ss=3D"gmail_attr">On Tue, Jun 4, 2024 at 3:15=E2=80=AFPM Ron Johnson &lt;<a=
 href=3D"mailto:ronljohnsonjr@gmail.com" target=3D"_blank">ronljohnsonjr@gm=
ail.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"=
margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-lef=
t:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><br></div><div class=3D"gmail_quot=
e"><div>But why tar instead of custom? That was part of my original questio=
n.</div></div></div></blockquote></div><div><br></div>I&#39;ve found it pre=
tty useful for programmatically accessing data in a dump for large database=
s outside of the normal pg_dump/pg_restore workflow. You don&#39;t have to =
seek through one large binary file to get to the data section to get at the=
 data.<br></div></blockquote><div><br></div><div>Interesting.=C2=A0 Please =
explain, though, since a big tarball _is_ &quot;one large binary file&quot;=
 that you have to sequentially scan.=C2=A0 (I don&#39;t know the internal s=
tructure of custom format files, and whether they have file pointers to eac=
h=C2=A0table.)</div></div></div></blockquote><div><br></div><div>Not if you=
 untar it first.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padd=
ing-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_quote"><div>Is it becaus=
e you need individual .dat=C2=A0&quot;COPY&quot; files for something other =
than loading into PG tables (since pg_restore --table=3Dxxxx does that, too=
), and directory format archives can be inconvenient?</div></div></div>
</blockquote></div><br>In the past I&#39;ve used it for data analysis outsi=
de of Postgres.<br><span class=3D"gmail_signature_prefix">-- </span><br><di=
v dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"ltr"><font face=3D"verd=
ana, sans-serif"><b>Gavin M. Roy</b><br>CTO</font><div><div>AWeber</div></d=
iv></div></div></div>

--000000000000def1b7061a254c24--