Date: Wed, 15 Jan 2025 08:30:20 +0000
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Nazir Bilal Yavuz <byavuz81@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>,
	Alvaro Herrera <alvherre@alvh.no-ip.org>,
	Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: per backend I/O statistics
Message-ID: <Z4dyHHxGfGK7bGjd@ip-10-97-1-34.eu-west-3.compute.internal>
References: <Z1_f2FzvUoL9KZsQ@paquier.xyz>
 <Z2BKTK9KX5Kc3HaU@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z2EZegYbH22LQHhN@paquier.xyz>
 <Z2EyxyEJPMMldj1t@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z2FAyFSg-lRGFZio@paquier.xyz>
 <Z2FF6ScWef9yIrJL@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z2JWUaY8AHBs99yp@paquier.xyz>
 <Z2KDy56Ls4nBl1Y8@ip-10-97-1-34.eu-west-3.compute.internal>
 <Z2OfYh3QJ7M5D0i-@paquier.xyz>
 <CAN55FZ0ZQTf-D+5Rup4H6zp+n3E2EuonYuzTiPkp=1GhesZU_A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAN55FZ0ZQTf-D+5Rup4H6zp+n3E2EuonYuzTiPkp=1GhesZU_A@mail.gmail.com>
Archived-At: <https://www.postgresql.org/message-id/Z4dyHHxGfGK7bGjd%40ip-10-97-1-34.eu-west-3.compute.internal>
Precedence: bulk

Hi,

On Wed, Jan 15, 2025 at 11:03:54AM +0300, Nazir Bilal Yavuz wrote:
> Hi,
> 
> On Thu, 19 Dec 2024 at 07:22, Michael Paquier <michael@paquier.xyz> wrote:
> >
> > Fixed that, bumped the two version counters, and done.
> 
> I encountered a problem while trying to add WAL stats to pg_stat_io
> and I wanted to hear your thoughts.
> 
> Right now, pgstat_prep_backend_pending() is called in both
> pgstat_count_io_op() and pgstat_count_io_op_time() to create a local
> PgStat_BackendPending entry. In that process,
> pgstat_prep_pending_entry() -> MemoryContextAllocZero() is called. The
> problem is that MemoryContextAllocZero() can not be called in the
> critical sections.
> 
> For example, here is what happens in the walsender backend:
> 
> '''
> ... ->
> exec_replication_command() ->
> SendBaseBackup() ->
> ...  ->
> XLogInsertRecord() ->
> START_CRIT_SECTION() /* Now we are in the critical section */ ->
> ... ->
> XLogWrite() ->
> pgstat_count_io_op_time() for the pg_pwrite() IO ->
> pgstat_prep_backend_pending() ->
> pgstat_prep_pending_entry() ->
> MemoryContextAllocZero() ->
> Failed at Assert("CritSectionCount == 0 || (context)->allowInCritSection")
> '''
> 
> With this commit it may not be possible to count IOs in the critical
> sections. I think the problem happens only if the local
> PgStat_BackendPending entry is being created for the first time for
> this backend in the critical section.

Yeah, I encountered the exact same thing and mentioned it in [1] (see R1.).

In [1]狢 did propose to use a new PendingBackendWalStats variable to "bypass" the
pgstat_prep_backend_pending() usage.

Michael mentioned in [2] that is not really consistent with the rest (what I
agree with) and that "we should rethink a bit the way pending entries are
retrieved". I did not think about it yet but that might be the way to
go, thoughts? 

[1]: https://www.postgresql.org/message-id/Z3zqc4o09dM/Ezyz%40ip-10-97-1-34.eu-west-3.compute.internal
[2]: https://www.postgresql.org/message-id/Z4dRlNuhSQ3hPPv2%40paquier.xyz

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com