Date: Tue, 11 Mar 2025 06:10:32 +0000
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: [BUG]: the walsender does not update its IO statistics until it
 exits
Message-ID: <Z8/T2DU208vq3rFc@ip-10-97-1-34.eu-west-3.compute.internal>
References: <Z73IsKBceoVd4t55@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z762lusJXp_wNF_z@paquier.xyz>
 <Z77jgvhwOu9S0a5r@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z8FMjlyNpNicucGa@paquier.xyz>
 <Z8GSY8mk+0qVHzsW@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z8ULFzNY24_SMCdu@paquier.xyz>
 <Z8WYf1jyy4MwOveQ@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z84xFrbmnW7Knb1Q@paquier.xyz>
 <Z878uhEaxYxRoXj7@ip-10-97-1-34.eu-west-3.compute.internal>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Z878uhEaxYxRoXj7@ip-10-97-1-34.eu-west-3.compute.internal>
Archived-At: 
 <https://www.postgresql.org/message-id/Z8/T2DU208vq3rFc%40ip-10-97-1-34.eu-west-3.compute.internal>
Precedence: bulk

Hi,

On Mon, Mar 10, 2025 at 02:52:42PM +0000, Bertrand Drouvot wrote:
> Hi,
> 
> On Mon, Mar 10, 2025 at 09:23:50AM +0900, Michael Paquier wrote:
> > On Mon, Mar 03, 2025 at 11:54:39AM +0000, Bertrand Drouvot wrote:
> > > So it does not look like what we're adding here can be seen as a primary bottleneck
> > > but that is probably worth implementing the "have_iostats" optimization attached.
> > > 
> > > Also, while I did not measure any noticeable extra lag, given the fact that 
> > > pgstat_flush_io() shows at about 5.5% and pgstat_flush_backend() at about 2.5%,
> > > that could still make sense to reduce the frequency of the flush calls, thoughts?
> > 
> > - I've spotted a third candidate which looks pretty solid, actually:
> > WalSndWaitForWal() before WalSndWait().  This leads to 2.9k reports in
> > the whole test suite, with much less contention in the reports.  These
> > can still be rather frequent, up to ~50 calls per seconds, but that's
> > really less than the two others.
> > 
> > Stats data is useful as long as it is possible to get an idea of how
> > the system behaves, particularly with a steady workload.  More
> > frequent reports are useful for spikey data detection, showing more
> > noise.  Still, too many reports may cause the part gathering the
> > reports to become a bottleneck, while we want it to offer hints about
> > bottlenecks.  So I would argue in favor of a more conservative choice
> > in the back branches than what the patch is proposing.
> 
> Yeah, fully agree. Anyway, having so many frequent stats reports makes little sense.
> 
> > Choice 3 i'm
> > quoting above is tempting by design: not too much, still frequent
> > enough to offer enough relevant information in the stats.
> 
> Yeah, I also agree that we should reduce the number of "reports". I'll look
> at the third option.

WalSndWaitForWal() is being used only for logical walsender. So we'd need to 
find another location for the physical walsender case. One option is to keep the
WalSndLoop() location and control the reports frequency.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com