From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: snapshot scalability: cache snapshots using a xact completion co
Date: Tue, 18 Aug 2020 16:45:32 -0700
Message-ID: <20200818234532.uiafo5br5lo6zhya@alap3.anarazel.de> (raw)
In-Reply-To: <20200818202805.5f7rbfh2zyfkzqqq@alap3.anarazel.de>
References: <E1k7tGP-0005V0-5k@gemulon.postgresql.org>
<20200818045246.GC3813@paquier.xyz>
<20200818051832.r2lixs4qi5whkqco@alap3.anarazel.de>
<1175198.1597728077@sss.pgh.pa.us>
<20200818202805.5f7rbfh2zyfkzqqq@alap3.anarazel.de>
On 2020-08-18 13:28:05 -0700, Andres Freund wrote:
> Hi,
>
> On 2020-08-18 01:21:17 -0400, Tom Lane wrote:
> > Andres Freund <andres@anarazel.de> writes:
> > > I'd written to Tom that I was planning to revert unless the number of
> > > failures were lower than initially indicated. But that actually seems to
> > > have come to pass (the failures are quicker to report because they don't
> > > run the subsequent tests, of course). I'd like to let the failures
> > > accumulate a bit longer, say until tomorrow Midday if I haven't figured
> > > it out by then. With the hope of finding some detail to help pinpoint
> > > the issue.
> >
> > There's certainly no obvious pattern here, so I agree with waiting for
> > more data.
>
> FWIW, I think I have found the bug, but I'm still working to reproduce
> the issue reliably enough that I can verify that the fix actually works.
>
> The issue is basically that 2PC PREPARE is weird, WRT procarray. The
> last snapshot built with GetSnapshotData() before the PREPARE doesn't
> include its own transaction in ->xip[], as normal. PrepareTransaction()
> removes the "normal" entry with ProcArrayClearTransaction(), which so
> far doesn't increase the xact completion count. Because the xact
> completion count is not increased, snapshots can be reused as long as
> they're taken before the 2PC transaction is finished. That's fine for
> other backends, but for the backend doing the PrepareTransaction() it's
> not, because there ->xip doesn't include the own backend.
>
> It's a bit tricky to reproduce exactly the issue the BF is occasionally
> hitting, because the way ->xmax is computed *limits* the
> damage. Combined with the use of SERIALIZABLE (preventing recomputation
> of the data snapshot) that makes it somewhat hard to hit.
I pushed a fix. After a while I figured out that it's not actually that
hard to test reliably. But it does require multiple sessions
interacting, particularly another session needs to acquire and commit a
transaction id that's later than the prepared transaction's.
I think it's worth adding an isolation test. But it doesn't seem like
extending prepared-transactions.spec makes too much sense, it doesn't
fit in well. It's a lot easier to reproduce the issue without
SERIALIZABLE, for example. Generally the file seems more about
serializable than 2PC...
So unless somebody disagrees I'm gonna add a new
prepared-transactions-2.spec.
Greetings,
Andres Freund
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: andres@anarazel.de, tgl@sss.pgh.pa.us, michael@paquier.xyz, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: snapshot scalability: cache snapshots using a xact completion co
In-Reply-To: <20200818234532.uiafo5br5lo6zhya@alap3.anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox