From: Ian Barwick <ian.barwick@2ndquadrant.com>
To: Andres Freund <andres@anarazel.de>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Daniel Gustafsson <daniel@yesql.se>
Cc: Jonathan S. Katz <jkatz@postgresql.org>
Cc: Alexander Korotkov <a.korotkov@postgrespro.ru>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Cc: Peter Geoghegan <pg@bowt.ie>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: David Rowley <dgrowleyml@gmail.com>
Subject: Re: Improving connection scalability: GetSnapshotData()
Date: Tue, 8 Sep 2020 13:23:01 +0900
Message-ID: <0006ffe6-6f76-c074-3e58-2801cee1c05f@2ndquadrant.com> (raw)
In-Reply-To: <20200908041114.5woxns33iv765fgh@alap3.anarazel.de>
References: <20200815164200.drok4tt5ju7ijivf@alap3.anarazel.de>
<20200816181604.l54m6kss5ntd6xow@alap3.anarazel.de>
<1006917.1597602624@sss.pgh.pa.us>
<1013484.1597609043@sss.pgh.pa.us>
<20200816203153.xqfvuevinjkweuez@alap3.anarazel.de>
<20200816205258.pjvmoljrk7ntqb3n@alap3.anarazel.de>
<20200816211146.nodohy6ck6mzfjwi@alap3.anarazel.de>
<20200816212657.3vyijoddr2yvgjjc@alap3.anarazel.de>
<20200903081554.GB8559@paquier.xyz>
<61291ffe-d611-f889-68b5-c298da9fb18f@2ndquadrant.com>
<20200908041114.5woxns33iv765fgh@alap3.anarazel.de>
On 2020/09/08 13:11, Andres Freund wrote:
> Hi,
>
> On 2020-09-08 13:03:01 +0900, Ian Barwick wrote:
(...)
>> I wonder if it's possible to increment "xactCompletionCount"
>> during replay along these lines:
>>
>> *** a/src/backend/access/transam/xact.c
>> --- b/src/backend/access/transam/xact.c
>> *************** xact_redo_commit(xl_xact_parsed_commit *
>> *** 5915,5920 ****
>> --- 5915,5924 ----
>> */
>> if (XactCompletionApplyFeedback(parsed->xinfo))
>> XLogRequestWalReceiverReply();
>> +
>> + LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
>> + ShmemVariableCache->xactCompletionCount++;
>> + LWLockRelease(ProcArrayLock);
>> }
>>
>> which seems to work (though quite possibly I've overlooked something I don't
>> know that I don't know about and it will all break horribly somewhere,
>> etc. etc.).
>
> We'd also need the same in a few more places. Probably worth looking at
> the list where we increment it on the primary (particularly we need to
> also increment it for aborts, and 2pc commit/aborts).
Yup.
> At first I was very confused as to why none of the existing tests have
> found this significant issue. But after thinking about it for a minute
> that's because they all use psql, and largely separate psql invocations
> for each query :(. Which means that there's no cached snapshot around...
>
> Do you want to try to write a patch?
Sure, I'll give it a go as I have some time right now.
Regards
Ian Barwick
--
Ian Barwick https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: ian.barwick@2ndquadrant.com, andres@anarazel.de, michael@paquier.xyz, tgl@sss.pgh.pa.us, robertmhaas@gmail.com, thomas.munro@gmail.com, alvherre@2ndquadrant.com, daniel@yesql.se, jkatz@postgresql.org, a.korotkov@postgrespro.ru, pg@bowt.ie, bruce@momjian.us, dgrowleyml@gmail.com
Subject: Re: Improving connection scalability: GetSnapshotData()
In-Reply-To: <0006ffe6-6f76-c074-3e58-2801cee1c05f@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox