Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nJ5ZJ-0004By-3V for pgsql-hackers@arkaria.postgresql.org; Sun, 13 Feb 2022 03:28:57 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1nJ5ZH-0006qC-W5 for pgsql-hackers@arkaria.postgresql.org; Sun, 13 Feb 2022 03:28:56 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nJ5ZF-0006q3-Tv for pgsql-hackers@lists.postgresql.org; Sun, 13 Feb 2022 03:28:55 +0000 Received: from out5-smtp.messagingengine.com ([66.111.4.29]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nJ5Z7-0002Et-Mi for pgsql-hackers@lists.postgresql.org; Sun, 13 Feb 2022 03:28:53 +0000 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 77A525C00E0; Sat, 12 Feb 2022 22:28:42 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Sat, 12 Feb 2022 22:28:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paquier.xyz; h= cc:cc:content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; bh=KfhYGTh7vQZdcnrKJEQ2i33TQHXe1qJxHlNjAR gmddY=; b=SOHb3Bo0Bf6tx7lnK4xRU5RIC8h7JRBo2i5/mT6jqloPleRFbbrgm4 nnfjSA7h4/QytaxfL/CzWajVMdAY0zaueOMQpvDVl2u2PEx1g1iwcxh2+wZYVIKB DKfQXux/rjQpjMaxNF51WvK1ywf4HBjmv6lLooLkc1nCpJDwG4ZjR51zJKpR4pth VtnSc1qY8Xwm8NSTYrw8CCyDQMhEkIIj7Z9GjHCnYDS5r9y3o5eXhupWgbBHXSY+ NWpcKYnpQTsZmpGFQ7cZ3hv9ACqr0GVANUYtr36AEsk3pihn7RC1Og9MNyTDyWJ+ agWsPiiIIqNwMSMly57OPl5u5OJ2iwwg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=KfhYGTh7vQZdcnrKJ EQ2i33TQHXe1qJxHlNjARgmddY=; b=nv6QY0hJMX3AlI+FdcNhU+XigE3EcXC/s HnkGrxOXYHkQ8Xo4eNBfJY13R+Q2iSiG5K3yv6XTz63dPG+dYl8eHVa7tjWzZcEE MENYb242CmIJO/HnY9wGWu9i2Fd9o3DbenRedQtWR0SOI9lvhTxIbnaBLcv4R4v7 z9Gj1DgZpjlVAPzaCmG+9zzPhkzQwen6PEqYaIHzmNeMezVI29jFIsfZkEe4p9fU 5vN9wiDrUrl1zpqgnDSF8HSeMFW/gG1ckrNO539e64Y3vWvQyT0mAoJY7VZHpWVi mXrpkK+RPYLTvBKwocvwVs4hxrEfMIW5NKfHG/6KCymqpXjAtSq2w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrieekgdegiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenfg hrlhcuvffnffculdejtddmnecujfgurhepfffhvffukfhfgggtuggjsehgtderredttddv necuhfhrohhmpefoihgthhgrvghlucfrrghquhhivghruceomhhitghhrggvlhesphgrqh huihgvrhdrgiihiieqnecuggftrfgrthhtvghrnhepvdegudeuhfdtueeltedtveejheeh ieevueeigeelteegleejleeiueeiheegvefhnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepmhhitghhrggvlhesphgrqhhuihgvrhdrgiihii X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 12 Feb 2022 22:28:40 -0500 (EST) Date: Sun, 13 Feb 2022 12:28:31 +0900 From: Michael Paquier To: Tom Lane Cc: Robert Haas , Andres Freund , PostgreSQL Hackers Subject: Re: pgsql: Add suport for server-side LZ4 base backup compression. Message-ID: References: <20220212220913.wnhgd7ak32srcmzo@alap3.anarazel.de> <2460218.1644704163@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="t1Ds/We4AwoKVbbW" Content-Disposition: inline In-Reply-To: <2460218.1644704163@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --t1Ds/We4AwoKVbbW Content-Type: multipart/mixed; boundary="nz4tkS9vxIoY01m4" Content-Disposition: inline --nz4tkS9vxIoY01m4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 12, 2022 at 05:16:03PM -0500, Tom Lane wrote: > It looks to me like somebody figured it didn't need any more support > than gzip/bzip2, which is wrong on a couple of grounds: > * hardly any modern platforms lack those, unlike lz4 > * we don't invoke either one of them during testing, only when > you explicitly ask to make a compressed tarball >=20 > I think adding an explicit PGAC_PATH_PROGS would be worth the cycles. Well, this somebody is I, and the buildfarm did not blow up on any of that so that looked rather fine. Adding a few cycles for this check is fine by me. What do you think of the attached? -- Michael --nz4tkS9vxIoY01m4 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="lz4-configure.patch" Content-Transfer-Encoding: quoted-printable diff --git a/src/bin/pg_basebackup/t/020_pg_receivewal.pl b/src/bin/pg_base= backup/t/020_pg_receivewal.pl index 0e6e685aa6..88dff2e673 100644 --- a/src/bin/pg_basebackup/t/020_pg_receivewal.pl +++ b/src/bin/pg_basebackup/t/020_pg_receivewal.pl @@ -130,8 +130,7 @@ SKIP: my $gzip =3D $ENV{GZIP_PROGRAM}; skip "program gzip is not found in your system", 1 if ( !defined $gzip - || $gzip eq '' - || system_log($gzip, '--version') !=3D 0); + || $gzip eq '' ); =20 my $gzip_is_valid =3D system_log($gzip, '--test', @zlib_wals); is($gzip_is_valid, 0, @@ -186,8 +185,7 @@ SKIP: my $lz4 =3D $ENV{LZ4}; skip "program lz4 is not found in your system", 1 if ( !defined $lz4 - || $lz4 eq '' - || system_log($lz4, '--version') !=3D 0); + || $lz4 eq '' ); =20 my $lz4_is_valid =3D system_log($lz4, '-t', @lz4_wals); is($lz4_is_valid, 0, diff --git a/configure b/configure index 0d52af5529..9305555658 100755 --- a/configure +++ b/configure @@ -650,6 +650,7 @@ CFLAGS_ARMV8_CRC32C CFLAGS_SSE42 have_win32_dbghelp LIBOBJS +LZ4 UUID_LIBS LDAP_LIBS_BE LDAP_LIBS_FE @@ -13832,6 +13833,60 @@ fi =20 fi =20 +if test -z "$LZ4"; then + for ac_prog in lz4 +do + # Extract the first word of "$ac_prog", so it can be a program name with= args. +set dummy $ac_prog; ac_word=3D$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_LZ4+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $LZ4 in + [\\/]* | ?:[\\/]*) + ac_cv_path_LZ4=3D"$LZ4" # Let the user override the test with a path. + ;; + *) + as_save_IFS=3D$IFS; IFS=3D$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=3D$as_save_IFS + test -z "$as_dir" && as_dir=3D. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_LZ4=3D"$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_= ext" >&5 + break 2 + fi +done + done +IFS=3D$as_save_IFS + + ;; +esac +fi +LZ4=3D$ac_cv_path_LZ4 +if test -n "$LZ4"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LZ4" >&5 +$as_echo "$LZ4" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + + test -n "$LZ4" && break +done + +else + # Report the value of LZ4 in configure's output in all cases. + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4" >&5 +$as_echo_n "checking for LZ4... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LZ4" >&5 +$as_echo "$LZ4" >&6; } +fi + if test "$with_lz4" =3D yes; then for ac_header in lz4.h do : diff --git a/configure.ac b/configure.ac index 2afc822b12..16167329fc 100644 --- a/configure.ac +++ b/configure.ac @@ -1485,6 +1485,7 @@ failure. It is possible the compiler isn't looking i= n the proper directory. Use --without-zlib to disable zlib support.])]) fi =20 +PGAC_PATH_PROGS(LZ4, lz4) if test "$with_lz4" =3D yes; then AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required= for LZ4])]) fi diff --git a/src/Makefile.global.in b/src/Makefile.global.in index 05c54b27de..9dcd54fcbd 100644 --- a/src/Makefile.global.in +++ b/src/Makefile.global.in @@ -350,7 +350,7 @@ XGETTEXT =3D @XGETTEXT@ =20 GZIP =3D gzip BZIP2 =3D bzip2 -LZ4 =3D lz4 +LZ4 =3D @LZ4@ =20 DOWNLOAD =3D wget -O $@ --no-use-server-timestamps #DOWNLOAD =3D curl -o $@ --nz4tkS9vxIoY01m4-- --t1Ds/We4AwoKVbbW Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmIIet8ACgkQnvQgOdby QH0JfQ//RAwsp3Dd2SuksQDjOBRl9apDWVhsAOQeKlXGYrV9+POLHuzDD3PTD7xe PVexe9VzPAWwI5Tgyt5Ycu4X8qUL8IikC/fmec2736/TbJ2raKXfagFszogFy+v4 XJC3cdIucI8kgENUy3JRqcAs3uFHuat54mdi8oDTKoXWZQPd0b5UwssrrH+XXhOY 2NamEQXy8yaDAtxOzHusYm7y9DDsZPlDVGDSXC1CvcESZXh7wLJBJhk8P5r1q4FU CaYfpJOo3+AtyW/HjS+U5jVo0VQJwwzNnt5CW5JwaAK96tjmA6UGaEKdzJpL2DJt Hvy0S8RoDRrp/3Apg7Lrq88rAqvldAsTnGNZx9TQ4hnlCmbtaODBYQ7Ldowl6BQb nXwMaPZRXuwh+nwkE73bNBUqxiw5fejGd0IumOvJpYt5E/gFnMqd0WGNriFDACbv uu8c6Zl+vYVaQ0McRJgrVITezb/f8rTp97SmQi5LhMoIJbjiK612/dMlDieCJVgG RRJzSIIRUm1+uJDUa9Nejf8FFtCbqIIbvh3bxU+DsUkuHBy5NiamuAqogzl0fhzG PSnxOF/VpdyiG/Ewdts0wK8adPZ0B42QDd04LpuohHpYg67z7o8hTffaYMb+Ra2Q VRNOjPjMFdP8alcpjy8e8OKc6Ezi1oNuw9doicf/dytBr475SYY= =JoqA -----END PGP SIGNATURE----- --t1Ds/We4AwoKVbbW--