Date: Mon, 8 Jun 2026 08:47:48 +0000
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Cc: Alexander Lakhin <exclusion@gmail.com>,
	"xunengzhou@gmail.com" <xunengzhou@gmail.com>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: t/035_standby_logical_decoding.pl might fail on attempt to read
 wrong timeline
Message-ID: <aiaBtENl7tTf2MM8@bdtpg>
References: <7daef094-abf3-4672-bc23-3df4763b16a3@gmail.com>
 <aiP/X1FThlZRCFiN@bdtpg>
 <OS9PR01MB1214908BA67A7811BD6281208F51C2@OS9PR01MB12149.jpnprd01.prod.outlook.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="HitlKQoyB8givWgr"
Content-Disposition: inline
In-Reply-To: 
 <OS9PR01MB1214908BA67A7811BD6281208F51C2@OS9PR01MB12149.jpnprd01.prod.outlook.com>
Archived-At: <https://www.postgresql.org/message-id/aiaBtENl7tTf2MM8%40bdtpg>
Precedence: bulk


--HitlKQoyB8givWgr
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Hi,

On Mon, Jun 08, 2026 at 04:25:45AM +0000, Hayato Kuroda (Fujitsu) wrote:
> Hi Alexander, Bertrand, Xuneng,
> 
> Thanks for seeing the failure. Our team also recognized but could not find the reason.
> 
> > Yeah, it looks like there is a race condition here. I think we should check if
> > the insertion timeline has already been set (like the walsummarizer is doing).
> 
> IIUC, the issue can happen if the walsender must read the WAL record generated
> after the promotion but the timeline could not be updated. 
> 
> However, I think logical_read_xlog_page() is called after the new WAL records
> are generated, i.e., am_cascading_walsender has already been false at that time.
> So not sure where is the race?

I ended up with this conclusion:

During promotion, there is a window where RecoveryInProgress() still
returns true but old timeline WAL segments have already been removed or
recycled by RemoveNonParentXlogFiles() in CleanupAfterArchiveRecovery().
This is because, in StartupXLOG(), WAL segments are cleaned up before
SharedRecoveryState transitions to RECOVERY_STATE_DONE.

If a walsender performing logical decoding calls logical_read_xlog_page()
during this window, it would get the old timeline from GetXLogReplayRecPtr(),
then attempt to open a WAL segment on that old timeline which no longer exists.

Attached:

0001: To fix this race

Fix by checking GetWALInsertionTimeLineIfSet() when RecoveryInProgress()
returns true. If InsertTimeLineID is already set (non-zero), the new timeline is
established and we use it directly, avoiding attempts to read from segments that
may have been removed.

0002: Adding a test in 035_standby_logical_decoding.pl

It makes use of a new injection point "promotion-after-wal-segment-cleanup" in
StartupXLOG(), right after CleanupAfterArchiveRecovery() removes old timeline
WAL segments but before SharedRecoveryState is set to RECOVERY_STATE_DONE.

The test fails without the fix in 0001 so it also somehow proves that the
diagnostic is right.

0003: Apply the same timeline fix to read_local_xlog_page_guts()

Indeed, it could hit the same race as mentioned by Xuneng-San.

0004: Add a test for 0003

Remark:

As far as the backpatching down to 16, it looks like 0001 to 0004 could be
backpatched as they are down to 17. For 16, we may want to also introduce
GetWALInsertionTimeLineIfSet().

I can have a closer look for the backpatch once we agree on how to fix those
races on master.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

--HitlKQoyB8givWgr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
	filename="v1-0001-Fix-race-condition-in-logical-decoding-timeline-s.patch"