pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Yushi Ogiwara <btogiwarayuushi@oss.nttdata.com>
To: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Pgsql Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Fix for consume_xids advancing XIDs incorrectly
Date: Wed, 30 Oct 2024 16:00:52 +0900
Message-ID: <1ea3663012d0cafb5639290eb19e3f71@oss.nttdata.com> (raw)
In-Reply-To: <da321d89-bc8b-4e6a-8ea6-248cf5ed4a50@oss.nttdata.com>
References: <2a62f138c9dcb6fcf5889b076d123aa2@oss.nttdata.com>
	<CAD21AoATuDuqSr=V3vCSbHbOg6g9zKL61EEbe4ZPG3xOUa0Pcw@mail.gmail.com>
	<50ee82a96ac0935de88f9a5ef021f04d@oss.nttdata.com>
	<CAD21AoAr92L0Yd-vF908He4B94F0W6Y1XC2ZUpm3WWoCLvONBw@mail.gmail.com>
	<57b588db6017ba20e67980fefa388b86@oss.nttdata.com>
	<a7667256-b98d-49a1-95a0-cca594151d07@oss.nttdata.com>
	<CAD21AoD8wJ0OiQbq=pc-TUCkEm1ohu6Et8G-Ntc8TFmPk3TuTA@mail.gmail.com>
	<88bed945-d8e3-4a28-ba9d-9eb3edde2d17@oss.nttdata.com>
	<CAD21AoB0uW34gHg23CoS=sQjgf29Fnj_7hF=VaCfcqexeBemMg@mail.gmail.com>
	<710b46d68d053eac91c9c8ec9feea09d@oss.nttdata.com>
	<da321d89-bc8b-4e6a-8ea6-248cf5ed4a50@oss.nttdata.com>

I made a new patch (skip_xid_correctly.diff) that incorporates the 
points we discussed:

1. Fix the issue that consume_xids consumes nxids+1 XIDs.
2. Update lastxid when calling GetTopFullTransactionId() to support 
nxids==1 case.
3. Forbid consume_xids when nxids==0.
4. Add comments explaining the return values of consume_xids and 
consume_xids_until, and the rationale for incrementing consumed++ when 
GetTopFullTransactionId() is called.

Also, I made another patch (support_blksz_32k.diff) that supports the 
block size == 32K case.

Best,
Yushi Ogiwara

Attachments: download all (2 files, .tar.gz)

skip_xid_correctly.diff (text/x-diff, 1.6K) download diff --git a/src/test/modules/xid_wraparound/xid_wraparound.c b/src/test/modules/xid_wraparound/xid_wraparound.c index dce81c0c6d..63f3305ee9 100644 --- a/src/test/modules/xid_wraparound/xid_wraparound.c +++ b/src/test/modules/xid_wraparound/xid_wraparound.c @@ -26,6 +26,7 @@ static FullTransactionId consume_xids_common(FullTransactionId untilxid, uint64 /* * Consume the specified number of XIDs. + * Returns the last XID assigned by this function. */ PG_FUNCTION_INFO_V1(consume_xids); Datum @@ -34,11 +35,8 @@ consume_xids(PG_FUNCTION_ARGS) int64 nxids = PG_GETARG_INT64(0); FullTransactionId lastxid; - if (nxids < 0) + if (nxids <= 0) elog(ERROR, "invalid nxids argument: %lld", (long long) nxids); - - if (nxids == 0) - lastxid = ReadNextFullTransactionId(); else lastxid = consume_xids_common(InvalidFullTransactionId, (uint64) nxids); @@ -47,6 +45,7 @@ consume_xids(PG_FUNCTION_ARGS) /* * Consume XIDs, up to the given XID. + * Returns the last XID assigned by this function. */ PG_FUNCTION_INFO_V1(consume_xids_until); Datum @@ -88,8 +87,15 @@ consume_xids_common(FullTransactionId untilxid, uint64 nxids) * GetNewTransactionId registers them in the subxid cache in PGPROC, until * the cache overflows, but beyond that, we don't keep track of the * consumed XIDs. + * + * If no top-level XID is assigned, a new one is obtained, + * and the consumed XID counter is incremented. */ - (void) GetTopTransactionId(); + if(!FullTransactionIdIsValid(GetTopFullTransactionIdIfAny())) + { + lastxid = GetTopFullTransactionId(); + consumed++; + } for (;;) {
support_blksz_32k.diff (text/x-diff, 1.6K) download diff --git a/src/test/modules/xid_wraparound/xid_wraparound.c b/src/test/modules/xid_wraparound/xid_wraparound.c index dce81c0c6d..95832fbecc 100644 --- a/src/test/modules/xid_wraparound/xid_wraparound.c +++ b/src/test/modules/xid_wraparound/xid_wraparound.c @@ -64,6 +64,14 @@ consume_xids_until(PG_FUNCTION_ARGS) PG_RETURN_FULLTRANSACTIONID(lastxid); } +/* + * These constants copied from .c files, because they're private. + */ +#define COMMIT_TS_XACTS_PER_PAGE (BLCKSZ / 10) +#define SUBTRANS_XACTS_PER_PAGE (BLCKSZ / sizeof(TransactionId)) +#define CLOG_XACTS_PER_BYTE 4 +#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE) + /* * Common functionality between the two public functions. */ @@ -115,7 +123,7 @@ consume_xids_common(FullTransactionId untilxid, uint64 nxids) * If we still have plenty of XIDs to consume, try to take a shortcut * and bump up the nextXid counter directly. */ - if (xids_left > 2000 && + if (xids_left > COMMIT_TS_XACTS_PER_PAGE && consumed - last_reported_at < REPORT_INTERVAL && MyProc->subxidStatus.overflowed) { @@ -153,14 +161,6 @@ consume_xids_common(FullTransactionId untilxid, uint64 nxids) return lastxid; } -/* - * These constants copied from .c files, because they're private. - */ -#define COMMIT_TS_XACTS_PER_PAGE (BLCKSZ / 10) -#define SUBTRANS_XACTS_PER_PAGE (BLCKSZ / sizeof(TransactionId)) -#define CLOG_XACTS_PER_BYTE 4 -#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE) - /* * All the interesting action in GetNewTransactionId happens when we extend * the SLRUs, or at the uint32 wraparound. If the nextXid counter is not close

view thread (19+ messages)  latest in thread

Message-ID: <1ea3663012d0cafb5639290eb19e3f71@oss.nttdata.com>
Permalink:  ../1ea3663012d0cafb5639290eb19e3f71@oss.nttdata.com/
Also on:    postgresql.org/message-id/1ea3663012d0cafb5639290eb19e3f71@oss.nttdata.com

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: btogiwarayuushi@oss.nttdata.com, masao.fujii@oss.nttdata.com, sawada.mshk@gmail.com
  Subject: Re: Fix for consume_xids advancing XIDs incorrectly
  In-Reply-To: <1ea3663012d0cafb5639290eb19e3f71@oss.nttdata.com>
  References: <2a62f138c9dcb6fcf5889b076d123aa2@oss.nttdata.com> <CAD21AoATuDuqSr=V3vCSbHbOg6g9zKL61EEbe4ZPG3xOUa0Pcw@mail.gmail.com> <50ee82a96ac0935de88f9a5ef021f04d@oss.nttdata.com> <CAD21AoAr92L0Yd-vF908He4B94F0W6Y1XC2ZUpm3WWoCLvONBw@mail.gmail.com> <57b588db6017ba20e67980fefa388b86@oss.nttdata.com> <a7667256-b98d-49a1-95a0-cca594151d07@oss.nttdata.com> <CAD21AoD8wJ0OiQbq=pc-TUCkEm1ohu6Et8G-Ntc8TFmPk3TuTA@mail.gmail.com> <88bed945-d8e3-4a28-ba9d-9eb3edde2d17@oss.nttdata.com> <CAD21AoB0uW34gHg23CoS=sQjgf29Fnj_7hF=VaCfcqexeBemMg@mail.gmail.com> <710b46d68d053eac91c9c8ec9feea09d@oss.nttdata.com> <da321d89-bc8b-4e6a-8ea6-248cf5ed4a50@oss.nttdata.com> <1ea3663012d0cafb5639290eb19e3f71@oss.nttdata.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox