public inbox for [email protected]  

help / color / mirror / Atom feed

From: Simon Riggs <[email protected]>
To: Andrey Borodin <[email protected]>
Cc: Pengchengliu <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: suboverflowed subtransactions concurrency performance optimize
Date: Tue, 30 Nov 2021 12:19:00 +0000
Message-ID: <CANbhV-GdpdpkvgAaRgaLsKQjTOdoGDXpCJkKU7wLrUBthTC_qQ@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>

On Mon, 30 Aug 2021 at 11:25, Andrey Borodin <[email protected]> wrote:
>
> Hi Pengcheng!
>
> You are solving important problem, thank you!
>
> > 30 авг. 2021 г., в 13:43, Pengchengliu <[email protected]> написал(а):
> >
> > To resolve this performance problem, we think about a solution which cache
> > SubtransSLRU to local cache.
> > First we can query parent transaction id from SubtransSLRU, and copy the
> > SLRU page to local cache page.
> > After that if we need query parent transaction id again, we can query it
> > from local cache directly.
>
> A copy of SLRU in each backend's cache can consume a lot of memory.

Yes, copying the whole SLRU into local cache seems overkill.

> Why create a copy if we can optimise shared representation of SLRU?

transam.c uses a single item cache to prevent thrashing from repeated
lookups, which reduces problems with shared access to SLRUs.
multitrans.c also has similar.

I notice that subtrans. doesn't have this, but could easily do so.
Patch attached, which seems separate to other attempts at tuning.

On review, I think it is also possible that we update subtrans ONLY if
someone uses >PGPROC_MAX_CACHED_SUBXIDS.
This would make subtrans much smaller and avoid one-entry-per-page
which is a major source of cacheing.
This would means some light changes in GetSnapshotData().
Let me know if that seems interesting also?

-- 
Simon Riggs                http://www.EnterpriseDB.com/


Attachments:

  [application/octet-stream] subtrans_single_item_cache.v1.patch (1.5K, ../CANbhV-GdpdpkvgAaRgaLsKQjTOdoGDXpCJkKU7wLrUBthTC_qQ@mail.gmail.com/2-subtrans_single_item_cache.v1.patch)
  download | inline diff:
diff --git a/src/backend/access/transam/subtrans.c b/src/backend/access/transam/subtrans.c
index 6a8e521f89..8aa3d9e53c 100644
--- a/src/backend/access/transam/subtrans.c
+++ b/src/backend/access/transam/subtrans.c
@@ -54,6 +54,14 @@
 #define TransactionIdToPage(xid) ((xid) / (TransactionId) SUBTRANS_XACTS_PER_PAGE)
 #define TransactionIdToEntry(xid) ((xid) % (TransactionId) SUBTRANS_XACTS_PER_PAGE)
 
+/*
+ * Single-item cache for results of SubTransGetTopmostTransaction.  It's worth having
+ * such a cache because we frequently find ourselves repeatedly checking the
+ * same XID, for example when scanning a table just after a bulk insert,
+ * update, or delete.
+ */
+static TransactionId cachedFetchXid = InvalidTransactionId;
+static TransactionId cachedFetchTopmostXid = InvalidTransactionId;
 
 /*
  * Link to shared-memory data structures for SUBTRANS control
@@ -155,6 +163,13 @@ SubTransGetTopmostTransaction(TransactionId xid)
 	/* Can't ask about stuff that might not be around anymore */
 	Assert(TransactionIdFollowsOrEquals(xid, TransactionXmin));
 
+	/*
+	 * Before going to the subtrans log, check our single item cache to
+	 * see if we know the result from a previous/recent request.
+	 */
+	if (TransactionIdEquals(xid, cachedFetchXid))
+		return cachedFetchTopmostXid;
+
 	while (TransactionIdIsValid(parentXid))
 	{
 		previousXid = parentXid;
@@ -174,6 +189,9 @@ SubTransGetTopmostTransaction(TransactionId xid)
 
 	Assert(TransactionIdIsValid(previousXid));
 
+	cachedFetchXid = xid;
+	cachedFetchTopmostXid = previousXid;
+
 	return previousXid;
 }
 

view thread (23+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: suboverflowed subtransactions concurrency performance optimize
  In-Reply-To: <CANbhV-GdpdpkvgAaRgaLsKQjTOdoGDXpCJkKU7wLrUBthTC_qQ@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox