Date: Fri, 5 Oct 2018 17:04:22 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: out-of-order XID insertion in KnownAssignedXids
Message-ID: <20181005080422.GA7863@paquier.xyz>
References: <0c96b653-4696-d4b4-6b5d-78143175d113@postgrespro.ru>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="tThc/1wpZn/ma/RB"
Content-Disposition: inline
In-Reply-To: <0c96b653-4696-d4b4-6b5d-78143175d113@postgrespro.ru>
Precedence: bulk


--tThc/1wpZn/ma/RB
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Oct 05, 2018 at 10:06:45AM +0300, Konstantin Knizhnik wrote:
> As you can notice, XID 2004495308 is encountered twice which cause error =
in
> KnownAssignedXidsAdd:
>=20
> =A0=A0=A0 if (head > tail &&
> =A0=A0=A0 =A0=A0=A0 TransactionIdFollowsOrEquals(KnownAssignedXids[head -=
 1], from_xid))
> =A0=A0=A0 {
> =A0=A0=A0 =A0=A0=A0 KnownAssignedXidsDisplay(LOG);
> =A0=A0=A0 =A0=A0=A0 elog(ERROR, "out-of-order XID insertion in KnownAssig=
nedXids");
> =A0=A0=A0 }
>=20
> The probability of this error is very small but it can quite easily
> reproduced: you should just set breakpoint in debugger after calling
> MarkAsPrepared in twophase.c and then try to prepare any transaction.
> MarkAsPrepared=A0 will add GXACT to proc array and at this moment there w=
ill
> be two entries in procarray with the same XID:
>
> [snip]
>
> Now generated RUNNING_XACTS record contains duplicated XIDs.

So, I have been doing exactly that, and if you trigger a manual
checkpoint then things happen quite correctly if you let the first
session finish:
rmgr: Standby     len (rec/tot):     58/    58, tx:          0, lsn:
0/016150F8, prev 0/01615088, desc: RUNNING_XACTS nextXid 608
latestCompletedXid 605 oldestRunningXid 606; 2 xacts: 607 606

If you still maintain the debugger after calling MarkAsPrepared, then
the manual checkpoint would block.  Now if you actually keep the
debugger, and wait for a checkpoint timeout to happen, then I can see
the incorrect record.  It is impressive that your customer has been able
to see that first, and then that you have been able to get into that
state with simple steps.

> I want to ask opinion of community about the best way of fixing this
> problem.  Should we avoid storing duplicated XIDs in procarray (by
> invalidating XID in original pgaxct) or eliminate/change check for
> duplicate in KnownAssignedXidsAdd (for example just ignore
> duplicates)?

Hmmmmm...  Please let me think through that first.  It seems to me that
the record should not be generated to begin with.  At least I am able to
confirm what you see.
--
Michael

--tThc/1wpZn/ma/RB
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAlu3GwYACgkQnvQgOdby
QH2eEQ/7B0suRBAnPglF2qH7A9EFc39GVaAZv7Zz/7Tu4bvtFBK4NqZyEzIBI3rY
G6gjtoWP4l3oOIdGjjMu6jap8DZ06RjCh0G57PDe+sdAAz/voDRD6xG8rmasrlyA
oLuI950/MzXhIDDydC2FrYbDjexGN/fhcshcA+b32HCxHn9r9Vz0MLmMoz0LqYRS
HzBGWX0VT1q+cPIS3JoYrBfV/v+par8HO6jW+xia9PdQ8ITYzlOLh6i1rubDF38p
2+mGAgBE2iCdnGS/2KjMXLg6/Sj2DaweWre5UJOch7UE5yXdL5QSQq7TdhGA5bAi
mPSLGtf8sFRBcHTjl7Uk4x+fWe/OsRfB9l9ASOSsAvO3py60x+Ft7nWyRNVAtWM4
yvBf5F9VJnhZY8ztUnSfnBHFYidhjqiDBeeMz6NP0bb3KH7hne2hI8xBbymQzzMh
dJ/r2XAZIuUoTyWSdedXCPFXfnluOEGBvOsNZwB/oPZCUUiprsye/qhva1ax31ST
fAxVlmybrSAHBx4w/eCIbqkBPtQcyaB3kqPU80QmQQ3w/Y79SWL9pvkL377IXw0R
q+WdPtTeYDT88yaU0Ln+duA+Gi5j6r35qNLAt1mbn1GfT4ZjZenItigfeSA/5evZ
t/HirCeGz9hjyI0J+S2r+Un1YUdetGIY/noEFjSWUnymTWKFklg=
=2ux9
-----END PGP SIGNATURE-----

--tThc/1wpZn/ma/RB--