Message-ID: <ec744999afcdbd6bec5d9e6572aba875060019cb.camel@j-davis.com>
Subject: Re: Minimal logical decoding on standbys
From: Jeff Davis <pgsql@j-davis.com>
To: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>, Andres Freund
	 <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, Thomas Munro
 <thomas.munro@gmail.com>,  Alvaro Herrera <alvherre@2ndquadrant.com>, Ibrar
 Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>,
 fabriziomello@gmail.com, tushar <tushar.ahuja@enterprisedb.com>, Rahila
 Syed <rahila.syed@2ndquadrant.com>,  pgsql-hackers
 <pgsql-hackers@postgresql.org>
Date: Wed, 01 Mar 2023 16:40:05 -0800
In-Reply-To: <061eeb26-156e-6011-02b6-b4b1422e2aa6@gmail.com>
References: 
	<CA+TgmoZ8-DXtDMMDGggFkSYyO8nmNdBKWjV6Ts-zYGR9p5sK5g@mail.gmail.com>
	 <CA+TgmobmTvEUDVOBzARa9eD9XLqsvDW+v29dV+WvDKZPJAwNTQ@mail.gmail.com>
	 <CA+TgmoaOsoQuXEo6gMq0GP8SyvCaS=ptYqdYc0uzDMoeShf_nw@mail.gmail.com>
	 <CA+TgmoYTTsxP8y6uknZvCBNCRq+1FJ4zGbX8Px1TGW459fGsaQ@mail.gmail.com>
	 <1f2ad9fd-53ff-74cc-a602-72d45423bcab@gmail.com>
	 <a78de011-81cf-729c-e110-e6d636fdca4d@gmail.com>
	 <05c0c4f3-82eb-bc6d-c876-5a265c05927b@gmail.com>
	 <CA+TgmoY8LAaTwAhS+YNQNrM+3K8eqxGvPT3gbJaHER86XShvFw@mail.gmail.com>
	 <20230106034036.2m4qnn7ep7b5ipet@awork3.anarazel.de>
	 <490206c5-bcde-e558-628a-dc67dae4d439@gmail.com>
	 <20230119024617.jzsgpcrgjdc6ysge@awork3.anarazel.de>
	 <a343b646-c632-4b9a-0e01-b661075d1d8b@gmail.com>
	 <b74520d5-5b9e-1477-2abc-95ba897f23e4@gmail.com>
	 <94d9d47e-72f4-393a-246e-e3fe8db7b741@gmail.com>
	 <a4c03fad-2986-ddab-c12e-ea2ee0119181@gmail.com>
	 <2d314c22b9e03415aa1c7d8fd1f698dae60effa7.camel@j-davis.com>
	 <061eeb26-156e-6011-02b6-b4b1422e2aa6@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu1 
MIME-Version: 1.0
Archived-At: <https://www.postgresql.org/message-id/ec744999afcdbd6bec5d9e6572aba875060019cb.camel%40j-davis.com>
Precedence: bulk

On Wed, 2023-03-01 at 11:51 +0100, Drouvot, Bertrand wrote:

>=20
> Why not "simply" call ConditionVariablePrepareToSleep() without any
> call to ConditionVariableTimedSleep() later?

ConditionVariableSleep() re-inserts itself into the queue if it was
previously removed. Without that, a single wakeup could remove it from
the wait queue, and the effects of ConditionVariablePrepareToSleep()
would be lost.

> In that case the walsender will be put in the wait queue (thanks to
> ConditionVariablePrepareToSleep())
> and will be waked up by the event on the socket, the timeout or the
> CV broadcast

I believe it will only be awakened once, and if it enters WalSndWait()
again, future ConditionVariableBroadcast/Signal() calls won't wake it
up any more.

>  (since IIUC they all rely on the same latch).

Relying on that fact seems like too much action-at-a-distance to me. If
we change the implementation of condition variables, then it would stop
working.

Also, since they are using the same latch, that means we are still
waking up too frequently, right? We haven't really solved the problem.

> That looks weird to use ConditionVariablePrepareToSleep() without
> actually using ConditionVariableTimedSleep()
> but it looks to me that it would achieve the same goal: having the
> walsender being waked up
> by the event on the socket, the timeout or the CV broadcast.

I don't think it actually works, because something needs to keep re-
inserting it into the queue after it gets removed. You could maybe hack
it to put ConditionVariablePrepareToSleep() *in* the loop, and never
sleep. But that just seems like too much of a hack, and I didn't really
look at the details to see if that would actually work.

To use condition variables properly, I think we'd need an API like
ConditionVariableEventsSleep(), which takes a WaitEventSet and a
timeout. I think this is what Andres was suggesting and seems like a
good idea. I looked into it and I don't think it's too hard to
implement -- we just need to WaitEventSetWait instead of WaitLatch.
There are a few details to sort out, like how to enable callers to
easily create the right WaitEventSet (it obviously needs to include
MyLatch, for instance) and update it with the right socket events.


--=20
Jeff Davis
PostgreSQL Contributor Team - AWS