From: Antonin Houska <ah@cybertec.at>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>,
    Andres Freund <andres@anarazel.de>,
    Alvaro Herrera <alvherre@alvh.no-ip.org>,
    Srinath Reddy Sadipiralla <srinath2133@gmail.com>,
    Matthias van de Meent <boekewurm+postgres@gmail.com>,
    Pg Hackers <pgsql-hackers@lists.postgresql.org>,
    Robert Treat <rob@xzilla.net>
Subject: Re: Adding REPACK [concurrently]
In-reply-to: <70574.1778512672@localhost>
References: 
 <CAA4eK1Jg21ODQ7fS2fvN5W_S5kDRhAP5inj3XMRQaa=s-GbYhw@mail.gmail.com>
 <202604071230.b5axxf3qna3m@alvherre.pgsql>
 <cdgw4sbbfcgk6du3iv54r2dgiy4tfywoklbotlmj4irxavdcr3@glxfw5jj277q>
 <227677.1775576304@localhost>
 <pveffyxhnuurhb44uzqlwo3rkyzorkfh2rot7uwzlf2axhfvbp@7nrs2omysxkc>
 <CAA4eK1JhuT5fyTosWDZ+Pgs+j7xEjObTyRMn80uNKgi_ivqHbw@mail.gmail.com>
 <CADzfLwWnbKcb3v8sStdgNE=WNc3uUqx5SiS4zftX2UaEfNzG5w@mail.gmail.com>
 <85813.1777901089@localhost> <27869.1777985266@localhost>
 <CAA4eK1KC6CGN-N2bUffSign8Sw4q6=8d3L-Xh4t+50GCdQb6zw@mail.gmail.com>
 <70574.1778512672@localhost>
Comments: In-reply-to Antonin Houska <ah@cybertec.at>
   message dated "Mon, 11 May 2026 17:17:52 +0200."
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Date: Mon, 11 May 2026 21:30:01 +0200
Message-ID: <82942.1778527801@localhost>
Archived-At: 
 <https://www.postgresql.org/message-id/82942.1778527801%40localhost>
Precedence: bulk

Antonin Houska <ah@cybertec.at> wrote:

> Amit Kapila <amit.kapila16@gmail.com> wrote:
>=20
> > On Tue, May 5, 2026 at 6:17=E2=80=AFPM Antonin Houska <ah@cybertec.at> =
wrote:
> > >
> > > Antonin Houska <ah@cybertec.at> wrote:
> > >
> > > I think the problem is that with database-specific snapshot,
> > > SnapBuildProcessRunningXacts() returns early, w/o adjusting builder->=
xmin
> > >
> > >         /*
> > >          * Database specific transaction info may exist to reach CONS=
ISTENT state
> > >          * faster, however the code below makes no use of it. Moreove=
r, such
> > >          * record might cause problems because the following normal (=
cluster-wide)
> > >          * record can have lower value of oldestRunningXid. In that c=
ase, let's
> > >          * wait with the cleanup for the next regular cluster-wide re=
cord.
> > >          */
> > >         if (OidIsValid(running->dbid))
> > >                 return;
> > >
> > > and thus some transactions whose XID is below running->oldestRunningX=
id may
> > > continue to be incorrectly considered running.
> > >
> > > I originally thought that this should not happen because such transac=
tions
> > > will be added to the builder's array of committed transactions by
> > > SnapBuildCommitTxn() anyway. However, I failed to notice that COMMIT =
record of
> > > a transaction listed in the xl_running_xacts WAL record is not guaran=
teed to
> > > follow the xl_running_xacts record in WAL. In other words, even if
> > > xl_running_xacts is created before a COMMIT record of the contained
> > > transaction, it may end up at higher LSN in WAL. So the cleanup I rel=
ied on
> > > might not take place.
> > >
> >=20
> > BTW, is it possible to write a test by using injection_points or via
> > manual steps (by using debugger, etc) so that we can more clearly
> > understand this problem and proposed fix?
>=20
> So far I could observe the situation in WAL, but have no idea how it can
> happen. For example, transaction 49242 gets committed here
>=20
> rmgr: Transaction len (rec/tot): 46/ 46, tx: 49242, lsn: 0/18BC28C8, prev
> 0/18BC2890, desc: COMMIT 2026-05-11 16:38:16.603265 CEST
>=20
> and then it appears in the 'xids' list of RUNNING_XACTS:
>=20
> rmgr: Standby     len (rec/tot):    106/   106, tx:          0, lsn:
> 0/18BC3140, prev 0/18BC3100, desc: RUNNING_XACTS nextXid 49255
> latestCompletedXid 49241 oldestRunningXid 49242; 13 xacts: 49248 49249 49=
246
> 49243 49252 49251 49244 49245 49242 49250 49253 49254 49247; dbid:5
>=20
>=20
> I thought the situation is quite common (and therefore nothing of
> SnapBuildProcessRunningXacts() should be skipped), but when trying to
> reproduce the problem, I noticed that LogStandbySnapshot() shouldn't allow
> that ordering issue when logical decoding is enabled:
>=20
> 	/*
> 	 * GetRunningTransactionData() acquired ProcArrayLock, we must release i=
t.
> 	 * For Hot Standby this can be done before inserting the WAL record
> 	 * because ProcArrayApplyRecoveryInfo() rechecks the commit status using
> 	 * the clog. For logical decoding, though, the lock can't be released
> 	 * early because the clog might be "in the future" from the POV of the
> 	 * historic snapshot. This would allow for situations where we're waiting
> 	 * for the end of a transaction listed in the xl_running_xacts record
> 	 * which, according to the WAL, has committed before the xl_running_xacts
> 	 * record. Fortunately this routine isn't executed frequently, and it's
> 	 * only a shared lock.
> 	 */
> 	if (!logical_decoding_enabled)
> 		LWLockRelease(ProcArrayLock);
>=20
> So I don't have the answer right now.

I think now that "waiting for the end of a transaction listed in the
xl_running_xacts record" in the comment is about transaction removal from
procarray. However, the COMMIT record can still be ahead of xl_running_xacts
because RecordTransactionCommit() is called before
ProcArrayEndTransaction(). I'll think again if the whole problem can be
reproduced with injection points.

--=20
Antonin Houska
Web: https://www.cybertec-postgresql.com