Date: Wed, 15 Jan 2025 10:55:44 +0000
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>,
	Alvaro Herrera <alvherre@alvh.no-ip.org>,
	Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
	pgsql-hackers@lists.postgresql.org,
	Andres Freund <andres@anarazel.de>
Subject: Re: per backend I/O statistics
Message-ID: <Z4eUMLFOjeKfW30u@ip-10-97-1-34.eu-west-3.compute.internal>
References: <Z2EZegYbH22LQHhN@paquier.xyz>
 <Z2EyxyEJPMMldj1t@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z2FAyFSg-lRGFZio@paquier.xyz>
 <Z2FF6ScWef9yIrJL@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z2JWUaY8AHBs99yp@paquier.xyz>
 <Z2KDy56Ls4nBl1Y8@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z2OfYh3QJ7M5D0i-@paquier.xyz>
 <CAN55FZ0ZQTf-D+5Rup4H6zp+n3E2EuonYuzTiPkp=1GhesZU_A@mail.gmail.com>
 <Z4dyHHxGfGK7bGjd@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z4d_eggsxtBEdJAG@paquier.xyz>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <Z4d_eggsxtBEdJAG@paquier.xyz>
Archived-At: <https://www.postgresql.org/message-id/Z4eUMLFOjeKfW30u%40ip-10-97-1-34.eu-west-3.compute.internal>
Precedence: bulk

Hi,

On Wed, Jan 15, 2025 at 06:27:22PM +0900, Michael Paquier wrote:
> On Wed, Jan 15, 2025 at 08:30:20AM +0000, Bertrand Drouvot wrote:
> > On Wed, Jan 15, 2025 at 11:03:54AM +0300, Nazir Bilal Yavuz wrote:
> >> With this commit it may not be possible to count IOs in the critical
> >> sections. I think the problem happens only if the local
> >> PgStat_BackendPending entry is being created for the first time for
> >> this backend in the critical section.
> > 
> > Yeah, I encountered the exact same thing and mentioned it in [1] (see R1.).
> > 
> > In [1]狢 did propose to use a new PendingBackendWalStats variable to "bypass" the
> > pgstat_prep_backend_pending() usage.
> > 
> > Michael mentioned in [2] that is not really consistent with the rest (what I
> > agree with) and that "we should rethink a bit the way pending entries are
> > retrieved". I did not think about it yet but that might be the way to
> > go, thoughts? 
> > 
> > [1]: https://www.postgresql.org/message-id/Z3zqc4o09dM/Ezyz%40ip-10-97-1-34.eu-west-3.compute.internal
> > [2]: https://www.postgresql.org/message-id/Z4dRlNuhSQ3hPPv2%40paquier.xyz
> 
> My problem is that this is not only related to backend stats, but to
> all variable-numbered stats kinds that require this behavior.

Yeah, agree that it's better to have a "generic" solution.

> A generic solution to the problem would be to allow
> pgStatPendingContext to have MemoryContextAllowInCriticalSection() set
> temporarily for the allocation of the pending data.
> Perhaps not for all stats kinds, just for these where we're OK with
> the potential memory footprint so we could have a flag in
> PgStat_KindInfo.  Giving to all stats kinds the responsibility to
> allocate a pending entry beforehand outside a critical section is
> another option, but that means going through different tweaks for all
> stats kinds that require them.
> 
> Andres, do you have any thoughts about that?  Not sure what would be 
> your view on this matter.  MemoryContextAllowInCriticalSection() for
> stats kinds that allow the case would be OK for me I guess,

Same here, that looks ok to me too.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com