From: Tom Lane <tgl@sss.pgh.pa.us>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: Alexander Lakhin <exclusion@gmail.com>
Cc: Sergei Kornilov <sk@zsrv.org>
Cc: Noah Misch <noah@leadboat.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: pgsql-bugs@lists.postgresql.org, Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: pgbf@twiska.com
Subject: Re: BUG #17928: Standby fails to decode WAL on termination of primary
Date: Mon, 25 Sep 2023 00:59:09 -0400
Message-ID: <86521.1695617949@sss.pgh.pa.us> (raw)
In-Reply-To: <CA+hUKGLF7NS-eUUuSaRoU7XvcTJDztf9wKXGo3GmjGnUrr8EzQ@mail.gmail.com>
References: <ZQpCG5QanJQXNVH-@paquier.xyz>
<CA+hUKGKJx=S77ttAtJ_iA=oBGUf2VTnfUp-gtJEhdXCbmdkoAA@mail.gmail.com>
<ZQ0R4kz3gN6bqS1N@paquier.xyz>
<CA+hUKGK1+3rj3c7L9HPCc8HajTVmkTx1CEkU5=rqgV3b4OvWeQ@mail.gmail.com>
<ZQ4r1vHcryBsSi_V@paquier.xyz>
<CA+hUKGKs=yj55ejAwuUJQuURU=uKjYgRqEQcCi1GrP2+=jUgNQ@mail.gmail.com>
<ZQ5tIoarrcwAlbVZ@paquier.xyz>
<CA+hUKGJf3Hhb2MB88-rW2di2H9XT0xr6-hd6ZjGEwdJs3A=b+Q@mail.gmail.com>
<ZQ9zf1QO8CP4TZRO@paquier.xyz>
<CA+hUKGKo5O_f8aBDoEMkxHmuWdFfmPPo6wL4q85b_0PDuHwZEA@mail.gmail.com>
<ZRDNNf8Etlvuo48a@paquier.xyz>
<CA+hUKG+cXwfAk0dEbD5CZ76p1uADGywuvb6_N4Uhziek54FZHg@mail.gmail.com>
<64446.1695611228@sss.pgh.pa.us>
<CA+hUKGLF7NS-eUUuSaRoU7XvcTJDztf9wKXGo3GmjGnUrr8EzQ@mail.gmail.com>
Thomas Munro <thomas.munro@gmail.com> writes:
> On Mon, Sep 25, 2023 at 4:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> That one is mine, so let me know if there's something particular
>> you'd like me to check.
> Thanks! But after spending the best part of a day installing and
> compiling stuff at snail's pace, I have just this moment managed to
> reproduce this reliably on an emulator running Debian armhf (= armv7).
> So hopefully I can track the issue down from here without bothering
> build farm owners.
FWIW, here's what I see on mamba:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 pg_comp_crc32c_sb8 (crc=3197562372, crc@entry=4294967295,
data=data@entry=0xfda6e030, len=<optimized out>) at pg_crc32c_sb8.c:56
56 uint32 a = *p4++ ^ crc;
(gdb) bt
#0 pg_comp_crc32c_sb8 (crc=3197562372, crc@entry=4294967295,
data=data@entry=0xfda6e030, len=<optimized out>) at pg_crc32c_sb8.c:56
#1 0x0195ae5c in ValidXLogRecord (state=0xfde5cc88, record=0xfda6e018,
recptr=<optimized out>) at xlogreader.c:1195
#2 0x0195cf18 in XLogDecodeNextRecord (state=state@entry=0xfde5cc88,
nonblocking=<optimized out>) at xlogreader.c:842
#3 0x0195d0c8 in XLogReadAhead (state=0xfde5cc88,
nonblocking=nonblocking@entry=false) at xlogreader.c:969
#4 0x01959760 in XLogPrefetcherNextBlock (pgsr_private=4259700408,
lsn=0xfda6b8a0) at xlogprefetcher.c:496
#5 0x0195a72c in lrq_prefetch (lrq=<optimized out>) at xlogprefetcher.c:256
#6 lrq_complete_lsn (lsn=50520048, lrq=0xfda6b858) at xlogprefetcher.c:294
#7 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0xfde5deb8,
errmsg=errmsg@entry=0xffffdd88) at xlogprefetcher.c:1041
#8 0x01960054 in ReadRecord (xlogprefetcher=0xfde5deb8, emode=emode@entry=15,
fetching_ckpt=fetching_ckpt@entry=false, replayTLI=replayTLI@entry=3)
at xlogrecovery.c:3067
#9 0x01963188 in PerformWalRecovery () at xlogrecovery.c:1756
#10 0x01952594 in StartupXLOG () at xlog.c:5470
#11 0x01c26e3c in StartupProcessMain () at startup.c:267
#12 0x01c1b504 in AuxiliaryProcessMain (auxtype=auxtype@entry=StartupProcess)
at auxprocess.c:141
#13 0x01c2276c in StartChildProcess (type=StartupProcess) at postmaster.c:5347
#14 0x01c260ec in PostmasterMain (argc=argc@entry=4,
argv=argv@entry=0xffffe1e4) at postmaster.c:1457
#15 0x01ee39d8 in main (argc=4, argv=0xffffe1e4) at main.c:198
(gdb) f 1
#1 0x0195ae5c in ValidXLogRecord (state=0xfde5cc88, record=0xfda6e018,
recptr=<optimized out>) at xlogreader.c:1195
1195 COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord);
(gdb) p *record
$3 = {xl_tot_len = 0, xl_xid = 0, xl_prev = 0, xl_info = 0 '\000',
xl_rmid = 0 '\000', xl_crc = 0}
Sure looks like ValidXLogRecord is assuming that record->xl_tot_len
can be trusted without reservation.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: tgl@sss.pgh.pa.us, thomas.munro@gmail.com, michael@paquier.xyz, exclusion@gmail.com, sk@zsrv.org, noah@leadboat.com, horikyota.ntt@gmail.com, masao.fujii@oss.nttdata.com, pgbf@twiska.com
Subject: Re: BUG #17928: Standby fails to decode WAL on termination of primary
In-Reply-To: <86521.1695617949@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox