Date: Sat, 12 Aug 2023 20:15:31 -0700
From: Noah Misch <noah@leadboat.com>
To: pgsql-hackers@postgresql.org
Subject: pg_waldump vs. all-zeros WAL files; server creation of such files
Message-ID: <20230813031531.GC2326466@rfd.leadboat.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="d6Gm4EdcadzBjdND"
Content-Disposition: inline
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: <https://www.postgresql.org/message-id/20230813031531.GC2326466%40rfd.leadboat.com>
Precedence: bulk


--d6Gm4EdcadzBjdND
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

The attached 010_zero.pl, when run as part of the pg_waldump test suite, fails
at today's master (c36b636) and v15 (1bc19df).  It passes at v14 (5a32af3).
Command "pg_waldump --start 0/01000000 --end 0/01000100" fails as follows:

  pg_waldump: error: WAL segment size must be a power of two between 1 MB and 1 GB, but the WAL file "000000010000000000000002" header specifies 0 bytes

Where it fails, the server has created an all-zeros WAL file under that name.
Where it succeeds, that file doesn't exist at all.  Two decisions to make:

- Should a clean server shutdown ever leave an all-zeros WAL file?  I think
  yes, it's okay to let that happen.
- Should "pg_waldump --start $X --end $Y" open files not needed for the
  requested range?  I think no.

Bisect of master got:
30a53b7 Wed Mar 8 16:56:37 2023 +0100 Allow tailoring of ICU locales with custom rules
Doesn't fail at $(git merge-base REL_15_STABLE master).  Bisect of v15 got:
811203d Sat Aug 6 11:50:23 2022 -0400 Fix data-corruption hazard in WAL-logged CREATE DATABASE.

I suspect those are innocent.  They changed the exact WAL content, which I
expect somehow caused creation of segment 2.

Oddly, I find only one other report of this:
https://www.postgresql.org/message-id/CAJ6DU3HiJ5FHbqPua19jAD%3DwLgiXBTjuHfbmv1jCOaNOpB3cCQ%40mail.gmail.com

Thanks,
nm

--d6Gm4EdcadzBjdND
Content-Type: application/x-perl
Content-Disposition: attachment; filename="010_zero.pl"
Content-Transfer-Encoding: quoted-printable

=0A# Copyright (c) 2021-2023, PostgreSQL Global Development Group=0A=0Ause =
strict;=0Ause warnings;=0Ause PostgreSQL::Test::Cluster;=0Ause PostgreSQL::=
Test::Utils;=0Ause Test::More;=0A=0Amy $node =3D PostgreSQL::Test::Cluster-=
>new('main');=0A$node->init;=0A=0Asub my_waldump()=0A{=0A	system('ls', $nod=
e->data_dir . "/pg_wal");=0A	command_ok([ 'pg_waldump', '-p', $node->data_d=
ir,=0A				 qw(--start 0/01000000 --end 0/01000100) ],=0A			   'runs');=0A}=
=0A=0A$node->start;=0A$node->command_ok([qw(createdb test)]);=0Amy_waldump;=
=0A$node->stop;=0Amy_waldump;=0A$node->start;=0Amy_waldump;=0A=0Adone_testi=
ng;=0A
--d6Gm4EdcadzBjdND--