Feedback-ID: i0fe9450f:Fastmail
Date: Tue, 6 Dec 2022 09:22:57 +0900
From: Michael Paquier <michael@paquier.xyz>
To: gkokolatos@pm.me
Cc: Justin Pryzby <pryzby@telsasoft.com>,
	pgsql-hackers@lists.postgresql.org,
	Rachel Heaton <rachelmheaton@gmail.com>
Subject: Re: Add LZ4 compression in pg_dump
Message-ID: <Y46LYQZltv7dx/jE@paquier.xyz>
References: 
 <O4mutIrCES8ZhlXJiMvzsivT7ztAMja2lkdL1LJx6O5f22I2W8PBIeLKz7mDLwxHoibcnRAYJXm1pH4tyUNC4a8eDzLn22a6Pb1S74Niexg=@pm.me>
 <Y4ao2viUjcO/c2Oc@paquier.xyz>
 <1j_ONBQqU3lQW7OqwbAFKbgcqjQosikcp7VPINqegLrHMCMGtFcXAuyjmW__VeviKyCbKTtucxFyPFUJ7sfqeOf4O90JEuUIe99Z5v7sdOg=@pm.me>
 <Y4gMAkNGIU1ShaoD@paquier.xyz>
 <jBiqIGoBbxyqLL2syvEDLm1W_TPukmBtAOkFyICkAssd8qupR7zjpA066Cj_syxQOVFsQORr3fc-g_WeATSXvdoybB-e7Xj26Y-C8U7rTKA=@pm.me>
 <Y4lbSoDPlVdbes1u@paquier.xyz>
 <T33dciWfZ3WaU10Mspm2_vsftrl0DI_cs4rQcrT_pLubaLq_HCRlW5QhEOT1cWtSRtdeTVXrL2r2BTYz3Fjz_3YNh6i77lwd5AxNxb5oanQ=@pm.me>
 <Y4q4SihPgXDVCL27@paquier.xyz>
 <Y42YMz/nnNj3fxXG@paquier.xyz>
 <DQn4czCWR1rcbGPLL7p3LfEr5-kGmlySm-H05VgroINdikvhtS5r9EdI6b8D8sjnbKdJ09k-cxs2AqijBeHAWk9Q8gvEAxPRHuLRhwONcGc=@pm.me>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="6PqK6/6g6hhte7X/"
Content-Disposition: inline
In-Reply-To: 
 <DQn4czCWR1rcbGPLL7p3LfEr5-kGmlySm-H05VgroINdikvhtS5r9EdI6b8D8sjnbKdJ09k-cxs2AqijBeHAWk9Q8gvEAxPRHuLRhwONcGc=@pm.me>
Archived-At: 
 <https://www.postgresql.org/message-id/Y46LYQZltv7dx/jE%40paquier.xyz>
Precedence: bulk


--6PqK6/6g6hhte7X/
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Mon, Dec 05, 2022 at 12:48:28PM +0000, gkokolatos@pm.me wrote:
> I also took the liberty of applying the test pattern when it the dump
> is explicitly compressed.

Sticking with glob_patterns is fine by me.

> I was thinking a bit more about this. I think that we can use the list
> TOC option of pg_restore. This option will first print out the header
> info which contains the compression. Perl utils already support to=20
> parse the generated output of a command. Please find an attempt to do
> so in the attached. The benefits of having some testing for this case
> become a bit more obvious in 0004 of the patchset, when lz4 is
> introduced.

This is where the fun is.  What you are doing here is more complete,
and we would make sure that the custom and data directory would always
see their contents compressed by default.  And it would have caught
the bug you mentioned upthread for the custom format.

I have kept things as you proposed at the end, added a few comments,
documented the new command_like and an extra command_like for
defaults_dir_format.  Glad to see this addressed, thanks!
--
Michael

--6PqK6/6g6hhte7X/
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmOOi2EACgkQnvQgOdby
QH1GHA//fhG9EwCOTcHSm2u7WtDVtU1cUqI901P/cao5kBxmz7iTf45c4fhcQIBk
wny9LeVbjjaYg6C0AFOdjadJjUNin8raWG0PwQSggPICdq0AKT/8SwXdI/sMdjOO
9SB+4c/1F6MBxs2PD9i0GI1XiPh7oS2MkSboKvC9QOZZgR+1kmSNqN2ouEFA0wQL
dr8q0H8eZaHSAKimbJ+JTL5BE71X9xvrTeJJReeP3fsXp7Ov2RvtTNsNUL11yRNt
vigDVublgT2BIY+f9L6AaIjDCF/jR6BBHBuH3nv3G5vCLPUGqTudEkTVdd7DshvU
y/OnbSD53ZdmfAXVF8B8dNI+oyZisjjb+7dYasvknrYdqkRvHw4cpsN/6OeKAxwq
PinEVh5anYGVlcuFLw8+Wtd7FzkWYq69pfQO2z5bvgdNIUO2G+S+lMn/jHB1734/
CwytPGclNaNTsNN2lRT3WGYsFyfg90KMXAaR1xc4t7fMQXFsNeYUH725EgOsbdGK
Q7a/FhMKKqeFImz47r4sS1p7tW++X8cfmpDVmew3XI9YeQFxOMPUNBqQtIMPBggS
1iCRNXZFmRt3PCULLDoNcKf8p4HHQxAk7XA1eWe3F2tfShNP7O0hfzVxtqGHUjyD
r1PdM3FythuRcwgLgqxzERY1OtigEr92v38xNAAt4XaNIrwBoUI=
=Ygx4
-----END PGP SIGNATURE-----

--6PqK6/6g6hhte7X/--