From: Michael Paquier <michael@paquier.xyz>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Alexander Lakhin <exclusion@gmail.com>
Cc: Sergei Kornilov <sk@zsrv.org>
Cc: Noah Misch <noah@leadboat.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: pgsql-bugs@lists.postgresql.org, Fujii Masao <masao.fujii@oss.nttdata.com>
Subject: Re: BUG #17928: Standby fails to decode WAL on termination of primary
Date: Sun, 24 Sep 2023 08:23:43 +0900
Message-ID: <ZQ9zf1QO8CP4TZRO@paquier.xyz> (raw)
In-Reply-To: <CA+hUKGJf3Hhb2MB88-rW2di2H9XT0xr6-hd6ZjGEwdJs3A=b+Q@mail.gmail.com>
References: <ee604ee6-f464-1502-0289-f1046c85c182@gmail.com>
<CA+hUKGKp_Go=EqA65r6naxrmXFfD-uH3DwPbm7tRCu9d2T=VyQ@mail.gmail.com>
<ZQpCG5QanJQXNVH-@paquier.xyz>
<CA+hUKGKJx=S77ttAtJ_iA=oBGUf2VTnfUp-gtJEhdXCbmdkoAA@mail.gmail.com>
<ZQ0R4kz3gN6bqS1N@paquier.xyz>
<CA+hUKGK1+3rj3c7L9HPCc8HajTVmkTx1CEkU5=rqgV3b4OvWeQ@mail.gmail.com>
<ZQ4r1vHcryBsSi_V@paquier.xyz>
<CA+hUKGKs=yj55ejAwuUJQuURU=uKjYgRqEQcCi1GrP2+=jUgNQ@mail.gmail.com>
<ZQ5tIoarrcwAlbVZ@paquier.xyz>
<CA+hUKGJf3Hhb2MB88-rW2di2H9XT0xr6-hd6ZjGEwdJs3A=b+Q@mail.gmail.com>
On Sun, Sep 24, 2023 at 09:48:42AM +1300, Thomas Munro wrote:
> "grison" has a little more detail -- we see
> pg_comp_crc32c_sb8(len=4294636456). I'm wondering how to reproduce
> this, but among the questions that jump out I have: why was it ever OK
> that we load record->xl_tot_len into total_len, perform header
> validation, determine that total_len < len (= this record is all on
> one page, no reassembly loop needed, so now we're in the single-page
> branch), then call ReadPageInternal() again, then call
> ValidXLogRecord() which internally loads record->xl_tot_len *again*?
> ReadPageInternal() might have changed xl_tot_len, no? That seems to
> be a possible pathway to reading past the end of the buffer in the CRC
> check, no?
>
> If that value didn't change underneath us, I think we'd need an
> explanation for how we finished up in the single-page branch at
> xlogreader.c:842 with a large xl_tot_len, which I'm not seeing yet,
> though it might take more coffee. (Possibly supporting the re-read
> theory is the fact that it's only happening on a few very slow
> computers, though I have no idea why it would only happen on master
> [so far at least].)
Hmm, it looks pretty clear that this is a HEAD-only thing as the
buildfarm shows and as you say, and my primary suspect here would be
71e4cc6b8ec6, I think. Any race condition underneath it would be
easier to see on slower machines. So it's likely possible that this
has messed up the page insertion logic.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../ZQ9zf1QO8CP4TZRO@paquier.xyz/2-signature.asc)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: michael@paquier.xyz, thomas.munro@gmail.com, exclusion@gmail.com, sk@zsrv.org, noah@leadboat.com, tgl@sss.pgh.pa.us, horikyota.ntt@gmail.com, masao.fujii@oss.nttdata.com
Subject: Re: BUG #17928: Standby fails to decode WAL on termination of primary
In-Reply-To: <ZQ9zf1QO8CP4TZRO@paquier.xyz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox