Message-ID: <061eeb26-156e-6011-02b6-b4b1422e2aa6@gmail.com>
Date: Wed, 1 Mar 2023 11:51:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.8.0
Subject: Re: Minimal logical decoding on standbys
To: Jeff Davis <pgsql@j-davis.com>, Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, Thomas Munro
 <thomas.munro@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>,
 Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar
 <amitdkhan.pg@gmail.com>, fabriziomello@gmail.com,
 tushar <tushar.ahuja@enterprisedb.com>,
 Rahila Syed <rahila.syed@2ndquadrant.com>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
References: <CA+TgmoZ8-DXtDMMDGggFkSYyO8nmNdBKWjV6Ts-zYGR9p5sK5g@mail.gmail.com>
 <CA+TgmobmTvEUDVOBzARa9eD9XLqsvDW+v29dV+WvDKZPJAwNTQ@mail.gmail.com>
 <CA+TgmoaOsoQuXEo6gMq0GP8SyvCaS=ptYqdYc0uzDMoeShf_nw@mail.gmail.com>
 <CA+TgmoYTTsxP8y6uknZvCBNCRq+1FJ4zGbX8Px1TGW459fGsaQ@mail.gmail.com>
 <1f2ad9fd-53ff-74cc-a602-72d45423bcab@gmail.com>
 <a78de011-81cf-729c-e110-e6d636fdca4d@gmail.com>
 <05c0c4f3-82eb-bc6d-c876-5a265c05927b@gmail.com>
 <CA+TgmoY8LAaTwAhS+YNQNrM+3K8eqxGvPT3gbJaHER86XShvFw@mail.gmail.com>
 <20230106034036.2m4qnn7ep7b5ipet@awork3.anarazel.de>
 <490206c5-bcde-e558-628a-dc67dae4d439@gmail.com>
 <20230119024617.jzsgpcrgjdc6ysge@awork3.anarazel.de>
 <a343b646-c632-4b9a-0e01-b661075d1d8b@gmail.com>
 <b74520d5-5b9e-1477-2abc-95ba897f23e4@gmail.com>
 <94d9d47e-72f4-393a-246e-e3fe8db7b741@gmail.com>
 <a4c03fad-2986-ddab-c12e-ea2ee0119181@gmail.com>
 <2d314c22b9e03415aa1c7d8fd1f698dae60effa7.camel@j-davis.com>
Content-Language: en-US
From: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>
In-Reply-To: <2d314c22b9e03415aa1c7d8fd1f698dae60effa7.camel@j-davis.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/061eeb26-156e-6011-02b6-b4b1422e2aa6%40gmail.com>
Precedence: bulk

Hi,

On 3/1/23 1:48 AM, Jeff Davis wrote:
> On Mon, 2023-02-27 at 09:40 +0100, Drouvot, Bertrand wrote:
>> Please find attached V51 tiny rebase due to a6cd1fc692 (for 0001) and
>> 8a8661828a (for 0005).
> 
> [ Jumping into this thread late, so I apologize if these comments have
> already been covered. ]
> 

Thanks for looking at it!

> Regarding v51-0004:
> 
> * Why is the CV sleep not being canceled?

I think that's an oversight, I'll look at it.

> * Comments on WalSndWaitForWal need to be updated to explain the
> difference between the flush (primary) and the replay (standby) cases.
> 

Yeah, will do.

> Overall, it seems like what you really want for the sleep/wakeup logic
> in WalSndWaitForLSN 

Typo for WalSndWaitForWal()?

> is something like this:
> 
>     condVar = RecoveryInProgress() ? replayCV : flushCV;
>     waitEvent = RecoveryInProgress() ?
>         WAIT_EVENT_WAL_SENDER_WAIT_REPLAY :
>         WAIT_EVENT_WAL_SENDER_WAIT_FLUSH;
> 
>     ConditionVariablePrepareToSleep(condVar);
>     for(;;)
>     {
>        ...
>        sleeptime = WalSndComputeSleepTime(GetCurrentTimestamp());
>        socketEvents = WL_SOCKET_READABLE;
>        if (pq_is_send_pending())
>            socketEvents = WL_SOCKET_WRITABLE;
>        ConditionVariableTimedSleepOrEvents(
>            condVar, sleeptime, socketEvents, waitEvent);
>     }
>     ConditionVariableCancelSleep();
> 
> 
> But the problem is that ConditionVariableTimedSleepOrEvents() doesn't
> exist, and I think that's what Andres was suggesting here[1].
> WalSndWait() only waits for a timeout or a socket event, but not a CV;
> ConditionVariableTimedSleep() only waits for a timeout or a CV, but not
> a socket event.
> 
> I'm also missing how WalSndWait() works currently. It calls
> ModifyWaitEvent() with NULL for the latch, so how does WalSndWakeup()
> wake it up?

I think it works because the latch is already assigned to the FeBeWaitSet
in pq_init()->AddWaitEventToSet() (for latch_pos).

> 
> Assuming I'm wrong, and WalSndWait() does use the latch, then I guess
> it could be extended by having two different latches in the WalSnd
> structure, and waking them up separately and waiting on the right one.

I'm not sure this is needed in this particular case, because:

Why not "simply" call ConditionVariablePrepareToSleep() without any call to ConditionVariableTimedSleep() later?

In that case the walsender will be put in the wait queue (thanks to ConditionVariablePrepareToSleep())
and will be waked up by the event on the socket, the timeout or the CV broadcast (since IIUC they all rely on the same latch).

So, something like:

	condVar = RecoveryInProgress() ? replayCV : flushCV;
	ConditionVariablePrepareToSleep(condVar);
	for(;;)
	{
            ...
            sleeptime = WalSndComputeSleepTime(GetCurrentTimestamp());
            socketEvents = WL_SOCKET_READABLE;
            if (pq_is_send_pending())
              socketEvents = WL_SOCKET_WRITABLE;
	   WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL);  <-- Note: the code within the loop does not change at all

      	}
      	ConditionVariableCancelSleep();


If the walsender is waked up by the CV broadcast, then it means the flush/replay occurred and then we should exit the loop
right after due to:

"
         /* check whether we're done */
         if (loc <= RecentFlushPtr)
             break;

"

meaning that in this particular case there is only one wake up due to the CV broadcast before exiting the loop.

That looks weird to use ConditionVariablePrepareToSleep() without actually using ConditionVariableTimedSleep()
but it looks to me that it would achieve the same goal: having the walsender being waked up
by the event on the socket, the timeout or the CV broadcast.

In that case we would be missing the WAIT_EVENT_WAL_SENDER_WAIT_REPLAY and/or the WAIT_EVENT_WAL_SENDER_WAIT_FLUSH
wait events thought (and we'd just provide the WAIT_EVENT_WAL_SENDER_WAIT_WAL one) but I'm not sure that's a big issue.

What do you think?

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com