From: Yushi Ogiwara <btogiwarayuushi@oss.nttdata.com>
To: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Pgsql Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Fix for consume_xids advancing XIDs incorrectly
Date: Wed, 30 Oct 2024 16:00:52 +0900
Message-ID: <1ea3663012d0cafb5639290eb19e3f71@oss.nttdata.com> (raw)
In-Reply-To: <da321d89-bc8b-4e6a-8ea6-248cf5ed4a50@oss.nttdata.com>
References: <2a62f138c9dcb6fcf5889b076d123aa2@oss.nttdata.com>
<CAD21AoATuDuqSr=V3vCSbHbOg6g9zKL61EEbe4ZPG3xOUa0Pcw@mail.gmail.com>
<50ee82a96ac0935de88f9a5ef021f04d@oss.nttdata.com>
<CAD21AoAr92L0Yd-vF908He4B94F0W6Y1XC2ZUpm3WWoCLvONBw@mail.gmail.com>
<57b588db6017ba20e67980fefa388b86@oss.nttdata.com>
<a7667256-b98d-49a1-95a0-cca594151d07@oss.nttdata.com>
<CAD21AoD8wJ0OiQbq=pc-TUCkEm1ohu6Et8G-Ntc8TFmPk3TuTA@mail.gmail.com>
<88bed945-d8e3-4a28-ba9d-9eb3edde2d17@oss.nttdata.com>
<CAD21AoB0uW34gHg23CoS=sQjgf29Fnj_7hF=VaCfcqexeBemMg@mail.gmail.com>
<710b46d68d053eac91c9c8ec9feea09d@oss.nttdata.com>
<da321d89-bc8b-4e6a-8ea6-248cf5ed4a50@oss.nttdata.com>
I made a new patch (skip_xid_correctly.diff) that incorporates the
points we discussed:
1. Fix the issue that consume_xids consumes nxids+1 XIDs.
2. Update lastxid when calling GetTopFullTransactionId() to support
nxids==1 case.
3. Forbid consume_xids when nxids==0.
4. Add comments explaining the return values of consume_xids and
consume_xids_until, and the rationale for incrementing consumed++ when
GetTopFullTransactionId() is called.
Also, I made another patch (support_blksz_32k.diff) that supports the
block size == 32K case.
Best,
Yushi Ogiwara
Attachments:download all (2 files, .tar.gz)skip_xid_correctly.diff (text/x-diff, 1.6K) downloaddiff --git a/src/test/modules/xid_wraparound/xid_wraparound.c b/src/test/modules/xid_wraparound/xid_wraparound.cindex dce81c0c6d..63f3305ee9 100644--- a/src/test/modules/xid_wraparound/xid_wraparound.c+++ b/src/test/modules/xid_wraparound/xid_wraparound.c@@ -26,6 +26,7 @@ static FullTransactionId consume_xids_common(FullTransactionId untilxid, uint64
/*
* Consume the specified number of XIDs.
+ * Returns the last XID assigned by this function.
*/
PG_FUNCTION_INFO_V1(consume_xids);
Datum
@@ -34,11 +35,8 @@ consume_xids(PG_FUNCTION_ARGS)
int64 nxids = PG_GETARG_INT64(0);
FullTransactionId lastxid;
- if (nxids < 0)+ if (nxids <= 0)
elog(ERROR, "invalid nxids argument: %lld", (long long) nxids);
-- if (nxids == 0)- lastxid = ReadNextFullTransactionId();
else
lastxid = consume_xids_common(InvalidFullTransactionId, (uint64) nxids);
@@ -47,6 +45,7 @@ consume_xids(PG_FUNCTION_ARGS)
/*
* Consume XIDs, up to the given XID.
+ * Returns the last XID assigned by this function.
*/
PG_FUNCTION_INFO_V1(consume_xids_until);
Datum
@@ -88,8 +87,15 @@ consume_xids_common(FullTransactionId untilxid, uint64 nxids)
* GetNewTransactionId registers them in the subxid cache in PGPROC, until
* the cache overflows, but beyond that, we don't keep track of the
* consumed XIDs.
+ * + * If no top-level XID is assigned, a new one is obtained,+ * and the consumed XID counter is incremented.
*/
- (void) GetTopTransactionId();+ if(!FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))+ {+ lastxid = GetTopFullTransactionId();+ consumed++;+ }
for (;;)
{
support_blksz_32k.diff (text/x-diff, 1.6K) downloaddiff --git a/src/test/modules/xid_wraparound/xid_wraparound.c b/src/test/modules/xid_wraparound/xid_wraparound.cindex dce81c0c6d..95832fbecc 100644--- a/src/test/modules/xid_wraparound/xid_wraparound.c+++ b/src/test/modules/xid_wraparound/xid_wraparound.c@@ -64,6 +64,14 @@ consume_xids_until(PG_FUNCTION_ARGS)
PG_RETURN_FULLTRANSACTIONID(lastxid);
}
+/*+ * These constants copied from .c files, because they're private.+ */+#define COMMIT_TS_XACTS_PER_PAGE (BLCKSZ / 10)+#define SUBTRANS_XACTS_PER_PAGE (BLCKSZ / sizeof(TransactionId))+#define CLOG_XACTS_PER_BYTE 4+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)+
/*
* Common functionality between the two public functions.
*/
@@ -115,7 +123,7 @@ consume_xids_common(FullTransactionId untilxid, uint64 nxids)
* If we still have plenty of XIDs to consume, try to take a shortcut
* and bump up the nextXid counter directly.
*/
- if (xids_left > 2000 &&+ if (xids_left > COMMIT_TS_XACTS_PER_PAGE &&
consumed - last_reported_at < REPORT_INTERVAL &&
MyProc->subxidStatus.overflowed)
{
@@ -153,14 +161,6 @@ consume_xids_common(FullTransactionId untilxid, uint64 nxids)
return lastxid;
}
-/*- * These constants copied from .c files, because they're private.- */-#define COMMIT_TS_XACTS_PER_PAGE (BLCKSZ / 10)-#define SUBTRANS_XACTS_PER_PAGE (BLCKSZ / sizeof(TransactionId))-#define CLOG_XACTS_PER_BYTE 4-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)-
/*
* All the interesting action in GetNewTransactionId happens when we extend
* the SLRUs, or at the uint32 wraparound. If the nextXid counter is not close
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-hackers@postgresql.org
Cc: btogiwarayuushi@oss.nttdata.com, masao.fujii@oss.nttdata.com, sawada.mshk@gmail.com
Subject: Re: Fix for consume_xids advancing XIDs incorrectly
In-Reply-To: <1ea3663012d0cafb5639290eb19e3f71@oss.nttdata.com>
References: <2a62f138c9dcb6fcf5889b076d123aa2@oss.nttdata.com> <CAD21AoATuDuqSr=V3vCSbHbOg6g9zKL61EEbe4ZPG3xOUa0Pcw@mail.gmail.com> <50ee82a96ac0935de88f9a5ef021f04d@oss.nttdata.com> <CAD21AoAr92L0Yd-vF908He4B94F0W6Y1XC2ZUpm3WWoCLvONBw@mail.gmail.com> <57b588db6017ba20e67980fefa388b86@oss.nttdata.com> <a7667256-b98d-49a1-95a0-cca594151d07@oss.nttdata.com> <CAD21AoD8wJ0OiQbq=pc-TUCkEm1ohu6Et8G-Ntc8TFmPk3TuTA@mail.gmail.com> <88bed945-d8e3-4a28-ba9d-9eb3edde2d17@oss.nttdata.com> <CAD21AoB0uW34gHg23CoS=sQjgf29Fnj_7hF=VaCfcqexeBemMg@mail.gmail.com> <710b46d68d053eac91c9c8ec9feea09d@oss.nttdata.com> <da321d89-bc8b-4e6a-8ea6-248cf5ed4a50@oss.nttdata.com> <1ea3663012d0cafb5639290eb19e3f71@oss.nttdata.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox