Subject: Re: pgsql: Implement pg_wal_replay_wait() stored procedure

From: Alexander Korotkov <[email-masked]>
Date: Mon, 4 Nov 2024 16:02:08 +0200

On Mon, Nov 4, 2024 at 8:04 AM Michael Paquier <[email-masked]> wrote:
>
> On Mon, Nov 04, 2024 at 06:29:42AM +0200, Alexander Korotkov wrote:
> > The attached patchset contains patch 0001, which improves handling of
> > not in recovery state by usage of PromoteIsTriggered().  When
> > (PromoteIsTriggered() == false), last replay LSN is not accepted and
> > not reported in errdetail().
> >
> > 0002 contains patch finishing implicit transaction in default
> > isolation level REPEATABLE READ or higher with revised commit message.
>
> I was just catching up with this thread, and I'm still confused about
> the state of things.  There are two things that are out of order for
> me, at least, after skimming through the code on HEAD (I suspect there
> is more):
> - WaitForLSNReplay() uses an initial set of checks that are duplicated
> in the main loop.  This is still overcomplicated, no?  Wouldn't it be
> simpler to eliminate the first of checks or just have a goto block
> with addLSNWaiter() called after the first round of checks?
> - pg_wal_replay_wait_status() returns a status based on a static
> variable that can only be accessed with the same backend as the one
> that has called the wait function.  That's.. Err.. Unfriendly.  Why
> being sticky with one backend for the job?

That's a bit awkward, but I think generally valid.  Some backend can
wait for LSN, then the result could be read from the same backend.
Difference backends could wait for different LSNs with different
results, I definitely don't feel like something should be shared
across backends.

> Using output parameters in a procedure is something I did not recall.
> Based on your point about not using a function due your argument based
> on the snapshots, let's just use that and forget about the status
> function entirely (please?).

Please, check [1].  Usage of output parameters is a bit awkward too,
because you need to pass some value in there.  And more importantly,
usage of output parameters also causes snapshot problem, as it causes
another snapshot to be held.

I'm tending to think we don't necessarily need to have ability to get
the waiting status in the initial version of this feature.  The work
on this feature started in 2016.  It's already very long for a simple
feature of just waiting for LSN (simple in general design, while our
snapshot reality makes one full of trouble).  Having in-core
implementation for this seems like breakthrough already, even if it
doesn't have all the abilities it could potentially have.

> Based on the latest set of issues reported, it does not feel like this
> is really baked and ready for prime day, either.  Perhaps it would be
> less confusing to just revert the whole and repost a more complete and
> structured implementation with an extra round of reviews?  There's
> still time in this release cycle.

Yes, this makes sense.  Will revert.

Links.
1. https://www.postgresql.org/message-id/CAPpHfdvRmTzGJw5rQdSMkTxUPZkjwtbQ%3DLJE2u9Jqh9gFXHpmg%40mail.gmail.com

------
Regards,
Alexander Korotkov
Supabase

---

From: Alvaro Herrera <[email-masked]>
Date: Mon, 4 Nov 2024 15:19:26 +0100

> another snapshot to be held.

I wonder if it would be better to go back to the original idea of using
special DDL syntax rather than a procedure.  It seems we've been piling
up hacks to get around the behavior of procedures, and we seem to have
grown one more to handle repeatable read transactions.

It's looking to me like there's just too much cruft in the quest to
avoid having to reach consensus on new syntax.  This might be a mistake.
Is it possible to backtrack on that decision?

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"Linux transformó mi computadora, de una `máquina para hacer cosas',
en un aparato realmente entretenido, sobre el cual cada día aprendo
algo nuevo" (Jaime Salinas)

---

From: Robert Haas <[email-masked]>
Date: Mon, 4 Nov 2024 10:53:32 -0500

> Is it possible to backtrack on that decision?

There's also the patch that Heikki posted to wait using a
protocol-level facility. Maybe that's just a better fit and we don't
need either a procedure or new syntax.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

---

From: Heikki Linnakangas <[email-masked]>
Date: Mon, 4 Nov 2024 17:57:19 +0200

On 04/11/2024 17:53, Robert Haas wrote:
> protocol-level facility.

It was Peter E

> Maybe that's just a better fit and we don't need either a procedure
> or new syntax.
I think it would still be good to expose the feature at SQL level too. 
Makes it easier to test and makes it usable without client library 
changes, for example.

-- 
Heikki Linnakangas
Neon (https://neon.tech)

---

From: Robert Haas <[email-masked]>
Date: Mon, 4 Nov 2024 11:53:20 -0500

> > There's also the patch that Heikki posted to wait using a

Oops.

> Makes it easier to test and makes it usable without client library

OK. Well then I agree with Álvaro. If using a procedure isn't working
out nicely, it's better to consider alternatives than to force a round
peg into a square hole (with due regard for the fact that you can fit
anything through the square hole if you make the square hole big
enough...).

---

From: Heikki Linnakangas <[email-masked]>
Date: Mon, 4 Nov 2024 19:11:58 +0200

> anything through the square hole if you make the square hole big

Agreed. Off the top of my, summary of issues I'm aware of:

- Missing documentation. The function itself is documented, but it's 
clearly intended for implementing read-after-write consistency in a 
cluster setup, so we should document how to do that.
- Not clear to me what the use case for the 'timeout' and 'noerror' 
arguments is. Why do we need it at all? (See first point on missing docs).
- Separate pg_wal_replay_wait_status() function to get result status is 
awkward
- Inconsistencies in behavior if server is not in recovery (Alexander's 
latest patch addresses this, but I didn't review it yet)
- pg_wal_replay_wait_status() is incorrectly marked as STABLE and 
PARALLEL SAFE
- Broken with default_transaction_isolation ='repeatable read'

Let's revert commit e546989a26 and rethink. I think the basic 
pg_wal_replay_wait() interface before e546989a26 was fine. Some of the 
above are issues above were present before e546989a26 already and need 
to be addressed in any case, but I believe they can be fixed without 
changing the interface.

-- 
Heikki Linnakangas
Neon (https://neon.tech)

---

From: Alexander Korotkov <[email-masked]>
Date: Mon, 4 Nov 2024 21:04:25 +0200

> Is it possible to backtrack on that decision?

Before committing pg_wal_replay_wait() I was under impression that
stored procedure would require the same amount of efforts than utility
statement to make backend "snapshot-less".  However, it appears that
if we have implicit REPEATABLE READ transaction, stored procedure
needs more efforts.  That makes the whole decision, at least,
questionable.

------
Regards,
Alexander Korotkov
Supabase

---

From: Alexander Korotkov <[email-masked]>
Date: Mon, 4 Nov 2024 23:05:00 +0200

> Makes it easier to test and makes it usable without client library

+1,
Also, it could potentially has wider use cases.  For example, waiting
for replay of not latest changes, but some intermediate changes.  Or
issuing pg_wal_replay_wait() from another process than one which made
the changes.

------
Regards,
Alexander Korotkov
Supabase