Feedback-ID: i0fe9450f:Fastmail
Date: Wed, 12 Jul 2023 08:41:45 +0900
From: Michael Paquier <michael@paquier.xyz>
To: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com>
Cc: pgsql-bugs <pgsql-bugs@lists.postgresql.org>
Subject: Re: Got FATAL in lock_twophase_recover() during recovery
Message-ID: <ZK3oueit2kXFZujm@paquier.xyz>
References: 
 <92895db6-5ae2-41f1-8038-302c31712691.mengjuan.cmj@alibaba-inc.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="1PFHOfH/V46NcEEo"
Content-Disposition: inline
In-Reply-To: 
 <92895db6-5ae2-41f1-8038-302c31712691.mengjuan.cmj@alibaba-inc.com>
Archived-At: 
 <https://www.postgresql.org/message-id/ZK3oueit2kXFZujm%40paquier.xyz>
Precedence: bulk


--1PFHOfH/V46NcEEo
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Jul 11, 2023 at 10:35:15AM +0800, suyu.cmj wrote:
> I want to report a bug about the recovery of two-phase transaction,
> in current implementation of crash recovery, there are two ways to
> recover 2pc data:=20
> 1=E3=80=81before redo, func restoreTwoPhaseData() will restore 2pc data
> those xid < ShmemVariableCache->nextXid, which is initialized from
> checkPoint.nextXid;=20
> 2=E3=80=81during redo, func xact_redo() will add 2pc from wal;
> The following scenario may cause the same 2pc transaction to be
> added repeatedly, I have attached a patch for pg11 that reproduces
> the error:=20
> 1=E3=80=81start creating checkpoint_1, checkpoint_1.redo is set as
> curInsert;=20
> 2=E3=80=81before set checkPoint_1.nextXid, a new 2pc is prepared, suppose
> the xid of this 2pc is 100, and then ShmemVariableCache->nextXid
> will be advanced as 101;=20
> 3=E3=80=81checkPoint_1.nextXid is set as 101;
> 4=E3=80=81in CheckPointTwoPhase() of checkpoint_1, 2pc_100 won't be copied
> to disk because its prepare_end_lsn > checkpoint_1.redo;=20
> 5=E3=80=81checkPoint_1 is finished, after checkpoint_timeout, start
> creating checkpoint_2;=20
> 6=E3=80=81during checkpoint_2, data of 2pc_100 will be copied to disk;
> 7=E3=80=81before UpdateControlFile() of checkpoint_2, crash happened;
> 8=E3=80=81during crash recovery, redo will start from checkpoint_1, and
> 2pc_100 will be restored first by restoreTwoPhaseData() because
> xid_100 < checkPoint_1.nextXid, which is 101; =20
> 9=E3=80=81because prepare_start_lsn of 2pc_100 > checkpoint_1.redo, 2pc_1=
00
> will be added again by xact_redo() during wal replay, resulting in
> the same 2pc data being added twice;=20

It looks like you have something here.  I'll try to look at it.  This
is a bug, so I have removed pgsql-hackers from the CC list keeping
only pgsql-bugs as cross-list posts are not encouraged.
--
Michael

--1PFHOfH/V46NcEEo
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmSt6LkACgkQnvQgOdby
QH29Vg/+PPY/fGywpdh6QU9/rYHnr0PGl+bDK0wad5ttOuui7dgk9x8VpD0Ax/NY
FT3k884JAXAHdx5L8t45lDI/Gdx47UYe5APwxzpJNd33/V+ShsJEOcRWr2TXGCKw
+SJU2FBigod3oefg6pMacNlENMbMwfhAGISBiNe8MJIwWU0xyaU+8valUMyZgxi7
MhQueRsYqahzCBUXkzSw/2ZcxzUzLZ4IgH3eZXyC0VaK8WhcKjdcfH0CqWLkehsR
bcoj8NJ55qH17XQn1JIYkJjWXOWxgjA1dzfk+HnkCiYIcV6moJ1TlW/CbWLFRItK
NbFqrk3LI5Bi8fcrvNBWxB9qRyi7WA2kyctkynl6Me3seOUfeL5CjZXIhfLfcMyD
T+2Lp/NCOWKl9lSwLF4XR/7Dn3zgt7mEaOlSbEeh65t+nBkZ8y5IuhKHatQHrzLz
/N2ZQyKmiIq7KxTle1JZqdeCno07cqZGgUIAnDbs+1mRG78skgX2uO3US6VyN1by
7KT00SPOJERfG79pMqL0VIxARoO2pR3JFsRHhEauxqe/YG2LJCSX6rsIcLG5oWSq
piiD8ZULRv3QnKjCnBQ++O6X4KniTpdfg+Jv8wpS4pj3jVpPNkmjIfccA8p7ohf0
jaR8Ha2S9KIsQrv/Vhwd5RlM2jwot3b1Ej3SQffvAv0h0Me5+y0=
=br1K
-----END PGP SIGNATURE-----

--1PFHOfH/V46NcEEo--