public inbox for [email protected]  

help / color / mirror / Atom feed

From: Tom Lane <[email protected]>
To: Michael Paquier <[email protected]>
Cc: [email protected]
Cc: Justin Pryzby <[email protected]>
Cc: Rachel Heaton <[email protected]>
Cc: Greg Stark <[email protected]>
Cc: [email protected]
Subject: Re: Add LZ4 compression in pg_dump
Date: Sun, 12 Oct 2025 13:24:37 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss=@protonmail.com>
	<PSMV5cBDnG2Wd93aYAyTE9RxCPpO0K7ZZQezDkqxwrDTrnp0R8fokD3N8JcmxmuHqAI4cRp_vMmPSUu1igCtaZfk-rLwoSWuBoFiCcRl8WQ=@pm.me>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<yTrMijGhpyxYVVeJuJPM5NandUq_5Qq647cqNuUpVkXSfbYPDj0VLfRK72jB2HWoeDgRIYIbjG7YbBrUuzv8fRF9Y5vALIrM9BGIgQLh8vM=@pm.me>
	<[email protected]>
	<bZeDkb5raMHiji1UIjvXjVIwc5qCHcdAb5h3CXXp85D8Mjwrm4kIGPh8UVbc2sJlfKgx2KsDHSrF35JAr-kFW5GJOx8YQlLKLv5vmAYLEjQ=@pm.me>
	<[email protected]>
	<syL-7HopUnzsy_Trvi6DON6tGwrcpujr-LhXo26AJ8AoP1fiGW66H33bMkPYU7tlu5HoqaSTt5FBeI6Ew1RLxnvTKy4-YZtBUJPgFMw3bQM=@pm.me>
	<[email protected]>

[ blast-from-the-past department ]

Michael Paquier <[email protected]> writes:
> At the end I am finishing with the attached.  I also saw an overlap
> with the addition of --jobs for the directory format vs not using the
> option, so I have removed the case where --jobs was not used in the
> directory format.

(This patch became commit 98fe74218.)

I am wondering if you remember why this bit:

+        # Give coverage for manually compressed blob.toc files during
+        # restore.
+        compress_cmd => {
+            program => $ENV{'GZIP_PROGRAM'},
+            args    => [ '-f', "$tempdir/compression_gzip_dir/blobs.toc", ],
+        },

was set up to manually compress blobs.toc but not the main TOC in the
toc.dat file.  It turns out that Gzip_read is broken for the case
of a zero-length read request [1], but we never reach that case
unless toc.dat is compressed.  We don't cover the getc_func member
of the compression stream API, either.

I thought for a bit about proposing that we compress toc.dat but not
blobs.toc, but that loses coverage in another way: the gets_func API
turns out to be used only while reading blobs.toc.  So we need to
compress both files manually if we want full coverage.

I think this change won't lose coverage, because there are other tests
in 002_pg_dump.pl that exercise directory format without extra
compression of anything.

Thoughts?

			regards, tom lane

[1] https://www.postgresql.org/message-id/3686.1760232320%40sss.pgh.pa.us

view thread (69+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Add LZ4 compression in pg_dump
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox