Date: Thu, 16 Jan 2025 17:11:09 +0000
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Michael Paquier <michael@paquier.xyz>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: per backend WAL statistics
Message-ID: <Z4k9rb3aTNVSmyq/@ip-10-97-1-34.eu-west-3.compute.internal>
References: <Z4BmdUQQh8jzlPHq@paquier.xyz>
 <Z4DrFhdlO6Xf13Xd@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z4dRlNuhSQ3hPPv2@paquier.xyz>
 <Z4ks4+wnAz8eEyX9@ip-10-97-1-34.eu-west-3.compute.internal>
 <xibh465r3x7xvpzaozu47fxule3dktvwz4yacce4pjlw76lkjd@fjvm56zbev37>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <xibh465r3x7xvpzaozu47fxule3dktvwz4yacce4pjlw76lkjd@fjvm56zbev37>
Archived-At: <https://www.postgresql.org/message-id/Z4k9rb3aTNVSmyq/%40ip-10-97-1-34.eu-west-3.compute.internal>
Precedence: bulk

Hi,

On Thu, Jan 16, 2025 at 11:38:47AM -0500, Andres Freund wrote:
> Hi,
> 
> On 2025-01-16 15:59:31 +0000, Bertrand Drouvot wrote:
> > On Wed, Jan 15, 2025 at 03:11:32PM +0900, Michael Paquier wrote:
> > > On Fri, Jan 10, 2025 at 09:40:38AM +0000, Bertrand Drouvot wrote:
> > > + * WAL pending statistics are incremented inside a critical section
> > > + * (see XLogWrite()), so we can't use pgstat_prep_pending_entry() and we rely on
> > > + * PendingBackendWalStats instead.
> > > + */
> > > +extern PGDLLIMPORT PgStat_PendingWalStats PendingBackendWalStats;
> > > 
> > > Hmm.  This makes me wonder if we should rethink a bit the way pending
> > > entries are retrieved and if we should do it beforehand for the WAL
> > > paths to avoid allocations in some critical sections.  Isn't that also
> > > because we avoid calling pgstat_prep_backend_pending() for the I/O
> > > case as only backends are supported now, discarding cases like the
> > > checkpointer where I/O could happen in a critical path?  As a whole,
> > > the approach taken by the patch is not really consistent with the
> > > rest.
> 
> > I agree that's better to have a generic solution and to be consistent with
> > the other variable-numbered stats. 
> > 
> > The attached is implementing in 0001 the proposition done in [1], i.e:
> > 
> > 1. It adds a new allow_critical_section to PgStat_KindInfo for pgstats kinds
> > 2. It ensures to set temporarly allowInCritSection to true when needed
> >
> > Note that for safety reason 0001 does set allowInCritSection back to false
> > unconditionally (means not checking again for allow_critical_section).
> 
> This is a preposterously bad idea.  The restriction to not allocate memory in
> critical sections exists for a reason,

Thanks for sharing your thoughts on it. In [1], you said:

"
My view is that for IO stats no memory allocation should be required - that
used to be the case and should be the case again
"

So, do you think that the initial proposal that has been made here (See R1. in
[2]) i.e make use of a new PendingBackendWalStats variable:

"
0003 does not rely on pgstat_prep_backend_pending() for its pending statistics
but on a new PendingBackendWalStats variable. The reason is that the pending wal
statistics are incremented in a critical section (see XLogWrite(), and so
a call to pgstat_prep_pending_entry() could trigger a failed assertion:
MemoryContextAllocZero()->"CritSectionCount == 0 || (context)->allowInCritSection"
"

and implemented up to v4 is a viable approach?

[1]: https://www.postgresql.org/message-id/66efowskppsns35v5u2m7k4sdnl7yoz5bo64tdjwq7r5lhplrz%40y7dme5xwh2r5
[2]: https://www.postgresql.org/message-id/Z3zqc4o09dM/Ezyz%40ip-10-97-1-34.eu-west-3.compute.internal

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com