Feedback-ID: i0fe9450f:Fastmail
Date: Mon, 14 Aug 2023 10:37:37 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Noah Misch <noah@leadboat.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: pg_waldump vs. all-zeros WAL files; server creation of such files
Message-ID: <ZNmFYUm2eRVtofyd@paquier.xyz>
References: <20230813031531.GC2326466@rfd.leadboat.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="3oiRuYNSWyQkNnfb"
Content-Disposition: inline
In-Reply-To: <20230813031531.GC2326466@rfd.leadboat.com>
Archived-At: <https://www.postgresql.org/message-id/ZNmFYUm2eRVtofyd%40paquier.xyz>
Precedence: bulk


--3oiRuYNSWyQkNnfb
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Sat, Aug 12, 2023 at 08:15:31PM -0700, Noah Misch wrote:
> The attached 010_zero.pl, when run as part of the pg_waldump test suite, =
fails
> at today's master (c36b636) and v15 (1bc19df).  It passes at v14 (5a32af3=
).
> Command "pg_waldump --start 0/01000000 --end 0/01000100" fails as follows:
>=20
>   pg_waldump: error: WAL segment size must be a power of two between
>   1 MB and 1 GB, but the WAL file "000000010000000000000002" header
>   specifies 0 bytes

So this depends on the ordering of the entries retrieved by readdir()
as much as the segments produced by the backend.

> Where it fails, the server has created an all-zeros WAL file under that n=
ame.
> Where it succeeds, that file doesn't exist at all.  Two decisions to make:
>=20
> - Should a clean server shutdown ever leave an all-zeros WAL file?  I thi=
nk
>   yes, it's okay to let that happen.

It doesn't hurt to leave that around.  On the contrary, it makes any
follow-up startup cheaper the bigger the segment size.

> - Should "pg_waldump --start $X --end $Y" open files not needed for the
>   requested range?  I think no.

So this is a case where identify_target_directory() is called with a
fname of NULL.  Agreed that search_directory could be smarter with the
files it should scan, especially if we have start and/or end LSNs at
hand to filter out what we'd like to be in the data folder.
--
Michael

--3oiRuYNSWyQkNnfb
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmTZhWEACgkQnvQgOdby
QH3ipBAApaVgVd/Z2uISnUmXVBdMnxpXQh7DxRoCNtDKsFwMQA8RqhdTPKVZDAp6
b1o6q0xSCvKi9aJs6s9LbLb+Nok5v7ZiipA83LsYy5VzImHd757Ix+VPQN911r2P
i/3rlT0WSDq9wCvGnob7YflCHgtZ7djWkwZbMox8txAO4bHe+0WusZCidp/8h99I
fgpFoc23F8HDNxkeMPhZUOkS19wVBUdBguabcRaFqjxv9vB3TcXYr4GKLiPb4j1O
ufGbBne01Ppo7+z9apeaJsSahGZfNlwLBYEXFk772MReMD2107Ue4yTFftsdlxWC
91OKrZuIBedQqQvMtbgFxQFEworQsyOHrs2TUgnMMDClBIWeuF2uIpkd4G5Xiu4o
eY3OAhWgcfPR8gibsMrf5Okv730lncLnPzWcW9EHoWHqZ5Fsj3zhq0XyLxzSYBdY
HtXyifVZjtOLxJT5iU7iM5X8GVYwp1fJu/5dOsuIZVXlRh2BvO2uhcBCl5olQ4vn
Ke9PHOL6j3znUitgzdgjfOnRzp/rGfsDYJvJ2YXmnw/IoVWZM6tazVyDsElR4tzN
ZJ7yoMdDwl3JJQoPUzkgEEHKdChkRu44M004a62qcB90haAFwHfyDG/TmFm5mWsj
PMQEmo7be5kMaVsRa3C/j+9Ob66WVYcgXiYDY58YR1bIr400Wzg=
=AEsq
-----END PGP SIGNATURE-----

--3oiRuYNSWyQkNnfb--