pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: gkokolatos@pm.me
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Introduce pg_receivewal gzip compression tests
Date: Fri, 16 Jul 2021 08:59:11 +0900
Message-ID: <YPDLz2x3o1aX2wRh@paquier.xyz> (raw)
In-Reply-To: <YPArqBhngZZkzs1x@paquier.xyz>
References: <YO1LTVgqpaY4jmN3@paquier.xyz>
	<20S0VOZrXJglfVMkn44Lks8F3gh66sHlxdes-d8XzjfcFV6VDHv9rF3_-KE2mnKTGlwoedEBz1k2UKrd2_hFBEQo2585MkrrJ-wpKWgGcBU=@pm.me>
	<YO1qYohoIRt2ssQP@paquier.xyz>
	<3NaNnzcFOo6mMA50bGVOS7RNlz1IqJ8nBxvMRQw9UZFUu9zSK2taKcmmrI2B0dehIXVskeCtbtA5yTnpkPGBTJvD-55DqTjRR-nji7tML68=@pm.me>
	<YO5H18MUJV+EMnR7@paquier.xyz>
	<icWOHz3LswZT-5mwtCbM8RHnhscksmm2gMw8BeejAoXyh9VWCiA2_jEpyGbv53HgrlZXf1F7Sb4nmxFeO5TxBTSRY6NH30RH83hDAgMHpZQ=@pm.me>
	<YO/coGVNAZTARxEw@paquier.xyz>
	<UEfDrva3Z-7dBiFYyj9Di0C-Ms1E2rbBCfgmgeP--2knm3ij4KddMN56vmtRlL_pp4CIYGhw_uaccoTjVBsTsk7xS4VHe-93TKg2tQb4JM0=@pm.me>
	<YPAdf9r5aJbDoHoq@paquier.xyz>
	<YPArqBhngZZkzs1x@paquier.xyz>

On Thu, Jul 15, 2021 at 09:35:52PM +0900, Michael Paquier wrote:
> For this one, I'll try to test harder on my own host.  I am curious to
> see if the other Windows members running the TAP tests have anything
> to say.  Looking at the code of zlib, this would come from gz_zero()
> in gzflush(), which could blow up on a write() in gz_comp().

bowerbird has just failed for the second time in a row on EACCESS, so
there is more here than meets the eye.  Looking at the code, I think I
have spotted what it is and the buildfarm logs give a very good hint:
# Running: pg_receivewal -D
:/prog/bf/root/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/t_020_pg_receivewal_primary_data/archive_wal
--verbose --endpos 0/3000028 --compress 1
pg_receivewal: starting log streaming at 0/2000000 (timeline 1)
pg_receivewal: fatal: could not fsync existing write-ahead log file
"000000010000000000000002.partial": Permission denied
not ok 20 - streaming some WAL using ZLIB compression

--compress is used and the sync fails for a non-compressed segment.
Looking at the code it is pretty obvious that open_walfile() is
getting confused with the handling of an existing .partial segment
while walmethods.c uses dir_data->compression in all the places that
matter.  So that's a legit bug, that happens only when mixing
pg_receivewal runs for where successive runs use the compression or
non-compression modes.

I am amazed that the other buildfarm members are not complaining, to
be honest.  jacana runs this TAP test with MinGW and ZLIB, and does
not complain.
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

view thread (29+ messages)  latest in thread

Message-ID: <YPDLz2x3o1aX2wRh@paquier.xyz>
Permalink:  ../YPDLz2x3o1aX2wRh@paquier.xyz/
Also on:    postgresql.org/message-id/YPDLz2x3o1aX2wRh@paquier.xyz

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: michael@paquier.xyz, gkokolatos@pm.me, pgsql-hackers@lists.postgresql.org
  Subject: Re: Introduce pg_receivewal gzip compression tests
  In-Reply-To: <YPDLz2x3o1aX2wRh@paquier.xyz>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox