Date: Tue, 11 Apr 2023 22:07:44 -0700
From: Noah Misch <noah@leadboat.com>
To: Andres Freund <andres@anarazel.de>
Cc: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>,
	Amit Kapila <amit.kapila16@gmail.com>,
	Alvaro Herrera <alvherre@alvh.no-ip.org>,
	Jeff Davis <pgsql@j-davis.com>, Robert Haas <robertmhaas@gmail.com>,
	Thomas Munro <thomas.munro@gmail.com>,
	Ibrar Ahmed <ibrar.ahmad@gmail.com>,
	Amit Khandekar <amitdkhan.pg@gmail.com>, fabriziomello@gmail.com,
	tushar <tushar.ahuja@enterprisedb.com>,
	Rahila Syed <rahila.syed@2ndquadrant.com>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Minimal logical decoding on standbys
Message-ID: <20230412050744.GA1217255@rfd.leadboat.com>
References: <71c3d91a-6b3b-5c0a-ffb9-630f17fe693e@gmail.com>
 <20230407075009.igg7be27ha2htkbt@awork3.anarazel.de>
 <1e923731-ed09-3621-c8b9-61f938b4c355@gmail.com>
 <20230407154757.ywqnldz4nsycap3g@awork3.anarazel.de>
 <20230407181226.6oyt4jcazy6eh7rx@awork3.anarazel.de>
 <20230411053657.GA1177147@rfd.leadboat.com>
 <7eb26a22-a6b2-f938-6d3c-6adcf4785004@gmail.com>
 <e217cdaf-53a1-6d41-59e2-4f4a8bbe9f23@gmail.com>
 <002cc876-0838-2821-e9d8-bb3791386814@gmail.com>
 <20230411201057.qsjw7pjgtpvxcvn2@awork3.anarazel.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230411201057.qsjw7pjgtpvxcvn2@awork3.anarazel.de>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: 
 <https://www.postgresql.org/message-id/20230412050744.GA1217255%40rfd.leadboat.com>
Precedence: bulk

On Tue, Apr 11, 2023 at 01:10:57PM -0700, Andres Freund wrote:
> On 2023-04-11 11:04:50 +0200, Drouvot, Bertrand wrote:
> > On 4/11/23 10:55 AM, Drouvot, Bertrand wrote:
> > > I think we might want to add:
> > > 
> > > $node_primary->wait_for_replay_catchup($node_standby);
> > > 
> > > before calling the slot creation.

> Pushed. Seems like a clear race in the test, so I didn't think it was worth
> waiting for testing it on hoverfly.

We'll see what happens in the next run.

> I think we should lower the log level, but perhaps wait for a few more cycles
> in case there are random failures?

Fine with me.

> I wonder if we should make the connections in poll_query_until to reduce
> verbosity - it's pretty annoying how much that can bloat the log. Perhaps also
> introduce some backoff? It's really annoying to have to trawl through all
> those logs when there's a problem.

Agreed.  My ranked wish list for poll_query_until is:

1. Exponential backoff
2. Closed-loop time control via Time::HiRes or similar, instead of assuming
   that ten loops complete in ~1s.  I've seen the loop take 3x as long as the
   intended timeout.
3. Connect less often than today's once per probe