Date: Thu, 15 Jul 2021 20:35:27 +0900
From: Michael Paquier <michael@paquier.xyz>
To: gkokolatos@pm.me
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Introduce pg_receivewal gzip compression tests
Message-ID: <YPAdf9r5aJbDoHoq@paquier.xyz>
References: <YOvkR0F6i4AmmmUe@paquier.xyz>
 <YO1C0THrAPCkZUXW@paquier.xyz>
 <YO1LTVgqpaY4jmN3@paquier.xyz>
 <20S0VOZrXJglfVMkn44Lks8F3gh66sHlxdes-d8XzjfcFV6VDHv9rF3_-KE2mnKTGlwoedEBz1k2UKrd2_hFBEQo2585MkrrJ-wpKWgGcBU=@pm.me>
 <YO1qYohoIRt2ssQP@paquier.xyz>
 <3NaNnzcFOo6mMA50bGVOS7RNlz1IqJ8nBxvMRQw9UZFUu9zSK2taKcmmrI2B0dehIXVskeCtbtA5yTnpkPGBTJvD-55DqTjRR-nji7tML68=@pm.me>
 <YO5H18MUJV+EMnR7@paquier.xyz>
 <icWOHz3LswZT-5mwtCbM8RHnhscksmm2gMw8BeejAoXyh9VWCiA2_jEpyGbv53HgrlZXf1F7Sb4nmxFeO5TxBTSRY6NH30RH83hDAgMHpZQ=@pm.me>
 <YO/coGVNAZTARxEw@paquier.xyz>
 <UEfDrva3Z-7dBiFYyj9Di0C-Ms1E2rbBCfgmgeP--2knm3ij4KddMN56vmtRlL_pp4CIYGhw_uaccoTjVBsTsk7xS4VHe-93TKg2tQb4JM0=@pm.me>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="2khNICuVq0+Np+4u"
Content-Disposition: inline
In-Reply-To: 
 <UEfDrva3Z-7dBiFYyj9Di0C-Ms1E2rbBCfgmgeP--2knm3ij4KddMN56vmtRlL_pp4CIYGhw_uaccoTjVBsTsk7xS4VHe-93TKg2tQb4JM0=@pm.me>
Archived-At: 
 <https://www.postgresql.org/message-id/YPAdf9r5aJbDoHoq%40paquier.xyz>
Precedence: bulk


--2khNICuVq0+Np+4u
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Thu, Jul 15, 2021 at 07:48:08AM +0000, gkokolatos@pm.me wrote:
> Let us hope that it will prevent some bugs from happening.

The buildfarm has two reports.

1) bowerbird on Windows/MSVC:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&dt=2021-07-15%2010%3A30%3A36
pg_receivewal: fatal: could not fsync existing write-ahead log file
"000000010000000000000002.partial": Permission denied
not ok 20 - streaming some WAL using ZLIB compression
I don't think the existing code can be blamed for that as this means a
failure with gzflush().  Likely a concurrency issue as that's an
EACCES.  If that's repeatable, that could point to an actual issue
with pg_receivewal --compress.

2) curculio:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=curculio&dt=2021-07-15%2010%3A30%3A15
# Running: gzip --test
 /home/pgbf/buildroot/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/t_020_pg_receivewal_primary_data/archive_wal/000000010000000000000002.gz
 /home/pgbf/buildroot/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/t_020_pg_receivewal_primary_data/archive_wal/000000010000000000000003.gz.partial
gzip:
 /home/pgbf/buildroot/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/t_020_pg_receivewal_primary_data/archive_wal/000000010000000000000003.gz.partial:
 unknown suffix: ignored
 not ok 24 - gzip verified the integrity of compressed WAL segments

Looking at the OpenBSD code (usr.bin/compress/main.c), long options
are supported, where --version does exit(0) without printing
anything, and --test is supported even if that's not on the man pages.
set_outfile() is doing a discard of the file suffixes it does not
recognize, and I think that their implementation bumps on .gz.partial
and generates an exit code of 512 to map with WARNING.  I still wish
to keep this test, and I'd like to think that the contents of
@zlib_wals are enough in terms of coverage.  What do you think?
--
Michael

--2khNICuVq0+Np+4u
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmDwHX8ACgkQnvQgOdby
QH1A4Q/+IJJjfBb563aZq8C1hJIvv58Eww2sR5jL/FBoJecsYQeyLua4QKuFTElj
xPmKsWzkTMrBw7seGF/QiBkB6rZHKPUz6xyF5wptkpqEmh6SlRXmtEttrg+Ru04B
3FBpMPsNupgFQaxzrJPuvBGdtZsuF/eBcyqS3r9mz6xn9mSMQGmOSMfkW8xbRSgV
JGH6/GUNiKwZkvsLuhx9iHhqOZ5J9F8brl/o7UOULMbIW/rfp8tunPXhsCBnoXKi
pyXzrrRtZ20fI+PmJBbPe8tOqDaB8lU++SE19hDGkL7ptM6m4HBAbQdVrz5uScAv
MNayCnI4xuVwNo4GwqK7uAVCinESkrYfNKx+FktZLfJHRdLTarZUK4gI+e1Hg6JD
ZfVZsnQTHgq7MA3GR0zExjqt7ptwa616r0tGnaFl+RL8wOjGZzwcCY13AvM4q/aK
sL7v5J3X0zxrNpEZ3sMTb2qYJe1l1s9aNSQHRjHistLGtS/gM24INlAYcjxd1QRZ
tp9bjtebqL8ezxtIZqF77eL7IR4wrCe2aWL1uDVaBKfEiZHZa4xuTrISO0HPqN+i
VIgywb34aQFQiyLM7jlsF1sBSpAM7sbXnAPPIMpE+uF06Yes/57Rnz002W4pQZCe
HRNuPq32XKnSRsDT3BAkzMNi4DnqjR5VlcF/6W2q+4Z0LlecQDs=
=yRVb
-----END PGP SIGNATURE-----

--2khNICuVq0+Np+4u--