public inbox for [email protected]
help / color / mirror / Atom feedFrom: Michael Paquier <[email protected]>
To: [email protected]
Cc: Justin Pryzby <[email protected]>
Cc: [email protected]
Cc: Rachel Heaton <[email protected]>
Subject: Re: Add LZ4 compression in pg_dump
Date: Fri, 2 Dec 2022 10:56:26 +0900
Message-ID: <[email protected]> (raw)
In-Reply-To: <jBiqIGoBbxyqLL2syvEDLm1W_TPukmBtAOkFyICkAssd8qupR7zjpA066Cj_syxQOVFsQORr3fc-g_WeATSXvdoybB-e7Xj26Y-C8U7rTKA=@pm.me>
References: <-pU80vtiyXMfSSpn4SWOR97t71MVmDOyHaJtvGsdEfroFsFUDdc-UaqD-8LMPMw2sy-7B2_IVnlrATfVvXwtv8n3kexLePZ7iWgiOnE4jRI=@pm.me>
<dWoK9v8RxFX0rEqRXwHPIAZ1JqOjgUJKJEQYKgXo180_Q1YJBN5GDUMYJyfYv_C8Vgs23ddTW0I2htMj2CFBPM1Rd-DFgyXXsNE5kwmyk3Q=@pm.me>
<[email protected]>
<O4mutIrCES8ZhlXJiMvzsivT7ztAMja2lkdL1LJx6O5f22I2W8PBIeLKz7mDLwxHoibcnRAYJXm1pH4tyUNC4a8eDzLn22a6Pb1S74Niexg=@pm.me>
<[email protected]>
<iKYD3Ryqsn88_UgYBiw3V803ISB5BJokmR2zsFtcedNVVeivIn_fqVv-oRRMqOpMiBFXsi72UYNMmuqMnEjjcQluTTpO-oxQWAVPeAlACnA=@pm.me>
<Y4ao2viUjcO/[email protected]>
<1j_ONBQqU3lQW7OqwbAFKbgcqjQosikcp7VPINqegLrHMCMGtFcXAuyjmW__VeviKyCbKTtucxFyPFUJ7sfqeOf4O90JEuUIe99Z5v7sdOg=@pm.me>
<[email protected]>
<jBiqIGoBbxyqLL2syvEDLm1W_TPukmBtAOkFyICkAssd8qupR7zjpA066Cj_syxQOVFsQORr3fc-g_WeATSXvdoybB-e7Xj26Y-C8U7rTKA=@pm.me>
On Thu, Dec 01, 2022 at 02:58:35PM +0000, [email protected] wrote:
> Nuking the warning from orbit and changing the behaviour around disabling
> the requested compression when the libraries are not present, should not
> mean that we need to change the behaviour of default values for different
> formats. Please find v13 attached which reinstates it.
Gah, thanks! And this default behavior is documented as dependent on
the compilation as well.
> Which in itself it got me looking and wondering why the tests succeeded.
> The only existing test covering that path is `defaults_dir_format` in
> `002_pg_dump.pl`. However as the test is currently written it does not
> check whether the output was compressed. The restore command would succeed
> in either case. A simple `gzip -t -r` against the directory will not
> suffice to test it, because there exist files which are never compressed
> in this format (.toc). A little bit more involved test case would need
> to be written, yet before I embark to this journey, I would like to know
> if you would agree to reinstate the defaults for those formats.
On top of my mind, I briefly recall that -r is not that portable. And
the toc format makes the files generated non-deterministic as these
use OIDs..
[.. thinks ..]
We are going to need a new thing here, as compress_cmd cannot be
directly used. What if we used only an array of glob()-able elements?
Let's say "expected_contents" that could include a "dir_path/*.gz"
conditional on $supports_gzip? glob() can only be calculated when the
test is run as the file names cannot be known beforehand :/
>> As per the patch, it is true that we do not need to bump the format of
>> the dump archives, as we can still store only the compression level
>> and guess the method from it. I have added some notes about that in
>> ReadHead and WriteHead to not forget.
>
> Agreed. A minor suggestion if you may.
>
> #ifndef HAVE_LIBZ
> - if (AH->compression != 0)
> + if (AH->compression_spec.algorithm == PG_COMPRESSION_GZIP)
> pg_log_warning("archive is compressed, but this installation does not support compression -- no data will be available");
> #endif
>
> It would seem a more consistent to error out in this case. We do error
> in all other cases where the compression is not available.
Makes sense.
I have gone through the patch again, and applied it. Thanks!
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../[email protected]/2-signature.asc)
download
view thread (65+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Add LZ4 compression in pg_dump
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox