pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: gkokolatos@pm.me
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Introduce pg_receivewal gzip compression tests
Date: Thu, 15 Jul 2021 20:35:27 +0900
Message-ID: <YPAdf9r5aJbDoHoq@paquier.xyz> (raw)
In-Reply-To: <UEfDrva3Z-7dBiFYyj9Di0C-Ms1E2rbBCfgmgeP--2knm3ij4KddMN56vmtRlL_pp4CIYGhw_uaccoTjVBsTsk7xS4VHe-93TKg2tQb4JM0=@pm.me>
References: <YOvkR0F6i4AmmmUe@paquier.xyz>
	<YO1C0THrAPCkZUXW@paquier.xyz>
	<YO1LTVgqpaY4jmN3@paquier.xyz>
	<20S0VOZrXJglfVMkn44Lks8F3gh66sHlxdes-d8XzjfcFV6VDHv9rF3_-KE2mnKTGlwoedEBz1k2UKrd2_hFBEQo2585MkrrJ-wpKWgGcBU=@pm.me>
	<YO1qYohoIRt2ssQP@paquier.xyz>
	<3NaNnzcFOo6mMA50bGVOS7RNlz1IqJ8nBxvMRQw9UZFUu9zSK2taKcmmrI2B0dehIXVskeCtbtA5yTnpkPGBTJvD-55DqTjRR-nji7tML68=@pm.me>
	<YO5H18MUJV+EMnR7@paquier.xyz>
	<icWOHz3LswZT-5mwtCbM8RHnhscksmm2gMw8BeejAoXyh9VWCiA2_jEpyGbv53HgrlZXf1F7Sb4nmxFeO5TxBTSRY6NH30RH83hDAgMHpZQ=@pm.me>
	<YO/coGVNAZTARxEw@paquier.xyz>
	<UEfDrva3Z-7dBiFYyj9Di0C-Ms1E2rbBCfgmgeP--2knm3ij4KddMN56vmtRlL_pp4CIYGhw_uaccoTjVBsTsk7xS4VHe-93TKg2tQb4JM0=@pm.me>

On Thu, Jul 15, 2021 at 07:48:08AM +0000, gkokolatos@pm.me wrote:
> Let us hope that it will prevent some bugs from happening.

The buildfarm has two reports.

1) bowerbird on Windows/MSVC:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&dt=2021-07-15%2010%3A30%3A36
pg_receivewal: fatal: could not fsync existing write-ahead log file
"000000010000000000000002.partial": Permission denied
not ok 20 - streaming some WAL using ZLIB compression
I don't think the existing code can be blamed for that as this means a
failure with gzflush().  Likely a concurrency issue as that's an
EACCES.  If that's repeatable, that could point to an actual issue
with pg_receivewal --compress.

2) curculio:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=curculio&dt=2021-07-15%2010%3A30%3A15
# Running: gzip --test
 /home/pgbf/buildroot/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/t_020_pg_receivewal_primary_data/archive_wal/000000010000000000000002.gz
 /home/pgbf/buildroot/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/t_020_pg_receivewal_primary_data/archive_wal/000000010000000000000003.gz.partial
gzip:
 /home/pgbf/buildroot/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/t_020_pg_receivewal_primary_data/archive_wal/000000010000000000000003.gz.partial:
 unknown suffix: ignored
 not ok 24 - gzip verified the integrity of compressed WAL segments

Looking at the OpenBSD code (usr.bin/compress/main.c), long options
are supported, where --version does exit(0) without printing
anything, and --test is supported even if that's not on the man pages.
set_outfile() is doing a discard of the file suffixes it does not
recognize, and I think that their implementation bumps on .gz.partial
and generates an exit code of 512 to map with WARNING.  I still wish
to keep this test, and I'd like to think that the contents of
@zlib_wals are enough in terms of coverage.  What do you think?
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

view thread (29+ messages)  latest in thread

Message-ID: <YPAdf9r5aJbDoHoq@paquier.xyz>
Permalink:  ../YPAdf9r5aJbDoHoq@paquier.xyz/
Also on:    postgresql.org/message-id/YPAdf9r5aJbDoHoq@paquier.xyz

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: michael@paquier.xyz, gkokolatos@pm.me, pgsql-hackers@lists.postgresql.org
  Subject: Re: Introduce pg_receivewal gzip compression tests
  In-Reply-To: <YPAdf9r5aJbDoHoq@paquier.xyz>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox