From: Tom Lane <tgl@sss.pgh.pa.us>
To: Michael Paquier <michael@paquier.xyz>
cc: gkokolatos@pm.me, Justin Pryzby <pryzby@telsasoft.com>,
        Rachel Heaton <rachelmheaton@gmail.com>, Greg Stark <stark@mit.edu>,
        pgsql-hackers@lists.postgresql.org
Subject: Re: Add LZ4 compression in pg_dump
In-reply-to: <YkucudXyfOwPOY5n@paquier.xyz>
References: 
 <faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss=@protonmail.com>
 <PSMV5cBDnG2Wd93aYAyTE9RxCPpO0K7ZZQezDkqxwrDTrnp0R8fokD3N8JcmxmuHqAI4cRp_vMmPSUu1igCtaZfk-rLwoSWuBoFiCcRl8WQ=@pm.me>
 <Yj6rXhLLW3h1vYt0@paquier.xyz> <20220326061441.GH28503@telsasoft.com>
 <YkK05K+0tjwmhLre@paquier.xyz>
 <yTrMijGhpyxYVVeJuJPM5NandUq_5Qq647cqNuUpVkXSfbYPDj0VLfRK72jB2HWoeDgRIYIbjG7YbBrUuzv8fRF9Y5vALIrM9BGIgQLh8vM=@pm.me>
 <YkPwmndklVKG4PE+@paquier.xyz>
 <bZeDkb5raMHiji1UIjvXjVIwc5qCHcdAb5h3CXXp85D8Mjwrm4kIGPh8UVbc2sJlfKgx2KsDHSrF35JAr-kFW5GJOx8YQlLKLv5vmAYLEjQ=@pm.me>
 <YkUTTK7dS6Zyeg9r@paquier.xyz>
 <syL-7HopUnzsy_Trvi6DON6tGwrcpujr-LhXo26AJ8AoP1fiGW66H33bMkPYU7tlu5HoqaSTt5FBeI6Ew1RLxnvTKy4-YZtBUJPgFMw3bQM=@pm.me>
 <YkucudXyfOwPOY5n@paquier.xyz>
Comments: In-reply-to Michael Paquier <michael@paquier.xyz>
	message dated "Tue, 05 Apr 2022 10:34:49 +0900"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <25344.1760289877.1@sss.pgh.pa.us>
Date: Sun, 12 Oct 2025 13:24:37 -0400
Message-ID: <25345.1760289877@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/25345.1760289877%40sss.pgh.pa.us>
Precedence: bulk

[ blast-from-the-past department ]

Michael Paquier <michael@paquier.xyz> writes:
> At the end I am finishing with the attached.  I also saw an overlap
> with the addition of --jobs for the directory format vs not using the
> option, so I have removed the case where --jobs was not used in the
> directory format.

(This patch became commit 98fe74218.)

I am wondering if you remember why this bit:

+        # Give coverage for manually compressed blob.toc files during
+        # restore.
+        compress_cmd => {
+            program => $ENV{'GZIP_PROGRAM'},
+            args    => [ '-f', "$tempdir/compression_gzip_dir/blobs.toc", ],
+        },

was set up to manually compress blobs.toc but not the main TOC in the
toc.dat file.  It turns out that Gzip_read is broken for the case
of a zero-length read request [1], but we never reach that case
unless toc.dat is compressed.  We don't cover the getc_func member
of the compression stream API, either.

I thought for a bit about proposing that we compress toc.dat but not
blobs.toc, but that loses coverage in another way: the gets_func API
turns out to be used only while reading blobs.toc.  So we need to
compress both files manually if we want full coverage.

I think this change won't lose coverage, because there are other tests
in 002_pg_dump.pl that exercise directory format without extra
compression of anything.

Thoughts?

			regards, tom lane

[1] https://www.postgresql.org/message-id/3686.1760232320%40sss.pgh.pa.us