Date: Mon, 10 Mar 2025 14:52:42 +0000
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: [BUG]: the walsender does not update its IO statistics until it
 exits
Message-ID: <Z878uhEaxYxRoXj7@ip-10-97-1-34.eu-west-3.compute.internal>
References: <Z73IsKBceoVd4t55@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z762lusJXp_wNF_z@paquier.xyz>
 <Z77jgvhwOu9S0a5r@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z8FMjlyNpNicucGa@paquier.xyz>
 <Z8GSY8mk+0qVHzsW@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z8ULFzNY24_SMCdu@paquier.xyz>
 <Z8WYf1jyy4MwOveQ@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z84xFrbmnW7Knb1Q@paquier.xyz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Z84xFrbmnW7Knb1Q@paquier.xyz>
Archived-At: 
 <https://www.postgresql.org/message-id/Z878uhEaxYxRoXj7%40ip-10-97-1-34.eu-west-3.compute.internal>
Precedence: bulk

Hi,

On Mon, Mar 10, 2025 at 09:23:50AM +0900, Michael Paquier wrote:
> On Mon, Mar 03, 2025 at 11:54:39AM +0000, Bertrand Drouvot wrote:
> > So it does not look like what we're adding here can be seen as a primary bottleneck
> > but that is probably worth implementing the "have_iostats" optimization attached.
> > 
> > Also, while I did not measure any noticeable extra lag, given the fact that 
> > pgstat_flush_io() shows at about 5.5% and pgstat_flush_backend() at about 2.5%,
> > that could still make sense to reduce the frequency of the flush calls, thoughts?
> 
> - I've spotted a third candidate which looks pretty solid, actually:
> WalSndWaitForWal() before WalSndWait().  This leads to 2.9k reports in
> the whole test suite, with much less contention in the reports.  These
> can still be rather frequent, up to ~50 calls per seconds, but that's
> really less than the two others.
> 
> Stats data is useful as long as it is possible to get an idea of how
> the system behaves, particularly with a steady workload.  More
> frequent reports are useful for spikey data detection, showing more
> noise.  Still, too many reports may cause the part gathering the
> reports to become a bottleneck, while we want it to offer hints about
> bottlenecks.  So I would argue in favor of a more conservative choice
> in the back branches than what the patch is proposing.

Yeah, fully agree. Anyway, having so many frequent stats reports makes little sense.

> Choice 3 i'm
> quoting above is tempting by design: not too much, still frequent
> enough to offer enough relevant information in the stats.

Yeah, I also agree that we should reduce the number of "reports". I'll look
at the third option.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com