Date: Sat, 19 Mar 2022 01:47:04 -0700
From: Noah Misch <noah@leadboat.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Thomas Munro <thomas.munro@gmail.com>,
	Alexander Lakhin <exclusion@gmail.com>,
	Andrew Dunstan <andrew@dunslane.net>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Why is src/test/modules/committs/t/002_standby.pl flaky?
Message-ID: <20220319084704.GB2822749@rfd.leadboat.com>
References: <1609152.1641691036@sss.pgh.pa.us>
 <c20eb022-f576-0cc5-d796-91ffaf10a2ca@gmail.com>
 <CA+hUKGKwiqnkmuj6tJ6E+wBDDDhB3d6_Lzmm3O=ZhRb9P8ETAA@mail.gmail.com>
 <CA+hUKG+9kOBvurZoouprt0it=7XutQR17NMpfTeyYJNyc6V_5w@mail.gmail.com>
 <3b904d7b-ef84-6f1b-9326-9f88c1374eb8@gmail.com>
 <CA+hUKGKaq1Nv79sQptmDCgJc2tGUKogW-YxAyTyF1WvJerR0RQ@mail.gmail.com>
 <CA+hUKGKPpbC2EM2tSy5-1cfj+-1d2PVwLU4kBNg8E+og+B3fbg@mail.gmail.com>
 <5d507424-13ce-d19f-2f5d-ab4c6a987316@gmail.com>
 <CA+hUKG+o_QzN1LrEPGJLa6JiZU4DpLRybkwZjdCErs2qsB6axg@mail.gmail.com>
 <2411751.1641849927@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2411751.1641849927@sss.pgh.pa.us>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: <https://www.postgresql.org/message-id/20220319084704.GB2822749%40rfd.leadboat.com>
Precedence: bulk

On Mon, Jan 10, 2022 at 04:25:27PM -0500, Tom Lane wrote:
> Apropos of that, it's worth noting that wait_for_catchup *is*
> dependent on up-to-date stats, and here's a recent run where
> it sure looks like the timeout cause is AWOL stats collector:
> 
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sungazer&dt=2022-01-10%2004%3A51%3A34
> 
> I wonder if we should refactor wait_for_catchup to probe the
> standby directly instead of relying on the upstream's view.

It would be nice.  For logical replication tests, do we have a monitoring API
independent of the stats collector?  If not and we don't want to add one, a
hacky alternative might be for wait_for_catchup to run a WAL-writing command
every ~20s.  That way, if the stats collector misses the datagram about the
standby reaching a certain LSN, the stats collector would have more chances.