MIME-Version: 1.0
References: 
 <CANzqJaAiTvMbuTUX86vZVCUY62KyHPffd_Dt1Po6Jmqv_9A_Bg@mail.gmail.com>
 <e31d0f6daaef9a7ce7248f43bd1063d444b4e859.camel@cybertec.at>
 <6bff91db-843e-4858-95dd-cc8bfb90c012@jakobs.com>
 <240b5c920d864da8157e65f8b7683c6a5e6d8aae.camel@cybertec.at>
 <CANzqJaAh+mPakfiEB2=GiNioMYRPef0eO7ex04-mmDLvoDPX0g@mail.gmail.com>
In-Reply-To: 
 <CANzqJaAh+mPakfiEB2=GiNioMYRPef0eO7ex04-mmDLvoDPX0g@mail.gmail.com>
From: Ron Johnson <ronljohnsonjr@gmail.com>
Date: Tue, 3 Feb 2026 11:33:04 -0500
Message-ID: 
 <CANzqJaBj0rjgXi2cX+imxGCRB02afDK7=PkrOaR6xg+iVm7qSg@mail.gmail.com>
Subject: Re: Does pg_dump custom format only compress the table data?
To: pgsql-admin <pgsql-admin@postgresql.org>
Content-Type: multipart/alternative; boundary="00000000000008c1b60649edff39"
Archived-At: 
 <https://www.postgresql.org/message-id/CANzqJaBj0rjgXi2cX%2BimxGCRB02afDK7%3DPkrOaR6xg%2BiVm7qSg%40mail.gmail.com>
Precedence: bulk

--00000000000008c1b60649edff39
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 3, 2026 at 11:16=E2=80=AFAM Ron Johnson <ronljohnsonjr@gmail.co=
m> wrote:

> On Tue, Feb 3, 2026 at 7:48=E2=80=AFAM Laurenz Albe <laurenz.albe@cyberte=
c.at>
> wrote:
>
>> On Tue, 2026-02-03 at 08:51 +0100, Holger Jakobs wrote:
>> > Am 03.02.26 um 07:06 schrieb Laurenz Albe:
>> > > On Mon, 2026-02-02 at 10:55 -0500, Ron Johnson wrote:
>> > > > Out of curiosity, I opened that dump file in vim, and saw not only
>> DDL strings but cleartext table data.
>> > > >
>> > > > Why isn't it compressed?
>> > > Are your DDL statement large enough that you would expect that
>> compressing
>> > > them would offer any real advantage?
>> >
>> > That wasn't the question. It was about why table DATA aren't
>> compressed,
>> > but in clear text.
>>
>> Ah, I see.
>>
>> By default, you wouldn't see any table data, because they get compressed=
.
>>
>> If a custom format dump is not compressed, the reason could be:
>>
>> - PostgreSQL was built without support for compression (--without-zlib)
>>
>> - pg_dump was called with --compress=3D0
>>
>
> I always install RPMs from postgresql.org, and in this case specified
> --compress=3Dzstd:long.
>
> $ psql -V
> psql (PostgreSQL) 17.7
>
> dba=3D# insert into do_not_delete (node_name) values ('foo');
> INSERT 0 1
> dba=3D# insert into do_not_delete (node_name) values ('bar');
> INSERT 0 1
> dba=3D# insert into do_not_delete (node_name) values ('blarge');
> INSERT 0 1
> dba=3D#
>
> $ pg_dump -V
> pg_dump (PostgreSQL) 17.7
>
> db=3Ddba
> $ pg_dump -Fc --compress=3Dzstd:long -v -d${db} -f ${db}.dump  2> ${db}.l=
og
> $ dir ${db}.dump
> -rw-r--r-- 1 postgres postgres 2205102870 2026-02-03 11:07:34 dba.dump
>
> $ strings ${db}.dump | grep -E 'foo|bar|blarge'
> foo     \N
> bar     \N
> blarge  \N
> \Afoo
> barYD
> [snip]
>

I tested more.  The default gzip compression does in fact compress
everything.  This must be a quirk of zstd.

--=20
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

--00000000000008c1b60649edff39
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Tue, Feb 3, 2026 at 11:16=E2=80=AFAM R=
on Johnson &lt;<a href=3D"mailto:ronljohnsonjr@gmail.com">ronljohnsonjr@gma=
il.com</a>&gt; wrote:</div><div class=3D"gmail_quote gmail_quote_container"=
><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border=
-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div di=
r=3D"ltr">On Tue, Feb 3, 2026 at 7:48=E2=80=AFAM Laurenz Albe &lt;<a href=
=3D"mailto:laurenz.albe@cybertec.at" target=3D"_blank">laurenz.albe@cyberte=
c.at</a>&gt; wrote:</div><div class=3D"gmail_quote"><blockquote class=3D"gm=
ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,=
204,204);padding-left:1ex">On Tue, 2026-02-03 at 08:51 +0100, Holger Jakobs=
 wrote:<br>
&gt; Am 03.02.26 um 07:06 schrieb Laurenz Albe:<br>
&gt; &gt; On Mon, 2026-02-02 at 10:55 -0500, Ron Johnson wrote:<br>
&gt; &gt; &gt; Out of curiosity, I opened that dump file in vim, and saw no=
t only DDL strings but cleartext table data.<br>
&gt; &gt; &gt; <br>
&gt; &gt; &gt; Why isn&#39;t it compressed?<br>
&gt; &gt; Are your DDL statement large enough that you would expect that co=
mpressing<br>
&gt; &gt; them would offer any real advantage?<br>
&gt; <br>
&gt; That wasn&#39;t the question. It was about why table DATA aren&#39;t c=
ompressed, <br>
&gt; but in clear text.<br>
<br>
Ah, I see.<br>
<br>
By default, you wouldn&#39;t see any table data, because they get compresse=
d.<br>
<br>
If a custom format dump is not compressed, the reason could be:<br>
<br>
- PostgreSQL was built without support for compression (--without-zlib)<br>
<br>
- pg_dump was called with --compress=3D0<br></blockquote><div><br></div><di=
v>I always install RPMs from <a href=3D"http://postgresql.org" target=3D"_b=
lank">postgresql.org</a>, and in this case specified --compress=3Dzstd:long=
.</div><div><br></div><div><font face=3D"monospace">$ psql -V<br>psql (Post=
greSQL) 17.7<br><br>dba=3D# insert into do_not_delete (node_name) values (&=
#39;foo&#39;);<br>INSERT 0 1<br>dba=3D# insert into do_not_delete (node_nam=
e) values (&#39;bar&#39;);<br>INSERT 0 1<br>dba=3D# insert into do_not_dele=
te (node_name) values (&#39;blarge&#39;);<br>INSERT 0 1<br>dba=3D# <br><br>=
$ pg_dump -V<br>pg_dump (PostgreSQL) 17.7<br><br>db=3Ddba<br>$ pg_dump -Fc =
--compress=3Dzstd:long -v -d${db} -f ${db}.dump =C2=A02&gt; ${db}.log<br>$ =
dir ${db}.dump<br>-rw-r--r-- 1 postgres postgres 2205102870 2026-02-03 11:0=
7:34 dba.dump<br><br>$ strings ${db}.dump | grep -E &#39;foo|bar|blarge&#39=
;<br>foo =C2=A0 =C2=A0 \N<br>bar =C2=A0 =C2=A0 \N<br>blarge =C2=A0\N<br>\Af=
oo<br>barYD<br>[snip]</font></div></div></div></blockquote><div>=C2=A0</div=
><div>I tested more.=C2=A0 The default gzip compression does in fact compre=
ss everything.=C2=A0 This must be a quirk of zstd.</div></div><div><br></di=
v><span class=3D"gmail_signature_prefix">-- </span><br><div dir=3D"ltr" cla=
ss=3D"gmail_signature"><div dir=3D"ltr">Death to &lt;Redacted&gt;, and butt=
er sauce.<div>Don&#39;t boil me, I&#39;m still alive.<br><div><div>&lt;Reda=
cted&gt; lobster!</div></div></div></div></div></div>

--00000000000008c1b60649edff39--