MIME-Version: 1.0
References: 
 <CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg@mail.gmail.com>
 <CAP53Pkw1RpyAVsvJwUOMgGnVoQUibMk+EB+w-8WqY8kzsC4WjA@mail.gmail.com>
 <CAP53Pkz-BmkLFJ+sAcr8H1sBPtWp8WYGzorH9yw+W5Tn0yH0sg@mail.gmail.com>
 <mflexfbmbkmql5snr322ldu72dfnjr2dntxx3ippaz4tcso7vl@zmxxgiws3qux>
 <CAP53PkzZ3UotnRrrnXWAv=F4avRq9MQ8zU+bxoN9tpovEu6fGQ@mail.gmail.com>
 <k2uxwancbi5blb2duny3kbuz667m47hqc4jv7xstlx3yyzcd3a@fjqww6avalt3>
 <CAHut+Put94CTpjQsqOJHdHkgJ2ZXq+qVSfMEcmDKLiWLW-hPfA@mail.gmail.com>
 <CAP53PkyOvXC7pWAiamvWth_JNeb=isrxX+PJT0pw_Hw5Czzf+Q@mail.gmail.com>
 <CAP53PkzGbyeJMLDAcvMRgzXPXYsYXZr3SBg0UwhfkYjqu8oK_g@mail.gmail.com>
 <CAP53PkxrmpECzVFpeeEEHDGe6u625s+YkmVv5-gw3L_NDSfbiA@mail.gmail.com>
 <CAN4CZFMon7XcTv+PKXB_ANUJ86k9VmJnx7BoW2q1m5Z2QQVZQw@mail.gmail.com>
 <CAP53PkxAS-rpp0kcOjg--q5NVNnRC+0kh1+gB8gnRNqacTqF5A@mail.gmail.com>
 <CAN4CZFNuxMqnmtujzemZ1i7hzYkaLD+BWPNq8tFXoPWp5R4How@mail.gmail.com>
 <CAP53PkxFP7i7-wy98ZmEJ11edYq-RrPvJoa4kzGhBBjERA4Nyw@mail.gmail.com>
 <06086cb4-881b-4f5a-96af-f275220ff52d@vondra.me>
 <CAN4CZFOdYhRfecTJjJZCwHtS-gzpSJ16YLQ7Wh0bXrg0=8keOw@mail.gmail.com>
 <CAP53PkxfVnQpcjrwegamPKdyy3RExXXeXNQW1ZWWtr=JtNsLNQ@mail.gmail.com>
 <CAN4CZFMk10YCixVTnt+uBaUH_BH1R=Er-PnPs2YDBhFjuE3K-Q@mail.gmail.com>
In-Reply-To: 
 <CAN4CZFMk10YCixVTnt+uBaUH_BH1R=Er-PnPs2YDBhFjuE3K-Q@mail.gmail.com>
From: Lukas Fittl <lukas@fittl.com>
Date: Wed, 18 Mar 2026 16:36:03 -0700
Message-ID: 
 <CAP53Pkwdg7Pmf+OqUBu4_SKgxEH3oosSrxxMc4v=mCqMUB049A@mail.gmail.com>
Subject: Re: Stack-based tracking of per-node WAL/buffer usage
To: Zsolt Parragi <zsolt.parragi@percona.com>
Cc: Tomas Vondra <tomas@vondra.me>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Andres Freund <andres@anarazel.de>,
	Peter Smith <smithpb2250@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CAP53Pkwdg7Pmf%2BOqUBu4_SKgxEH3oosSrxxMc4v%3DmCqMUB049A%40mail.gmail.com>
Precedence: bulk

On Wed, Mar 18, 2026 at 1:49=E2=80=AFPM Zsolt Parragi <zsolt.parragi@percon=
a.com> wrote:
>
> + instr_stack.stack_space *=3D 2;
> + instr_stack.entries =3D repalloc_array(instr_stack.entries,
> Instrumentation *, instr_stack.stack_space);
>
> Can't this also cause issues with OOM? repalloc_array failing, but we
> already doubled stack_space.
> The initialization above uses the same order, but that should be safe
> as entries is initially NULL.

Yeah, that's fair, I think its reasonable to do the update of
instr_stack.stack_space after the allocation succeeded. I also think
its good to do that even for the initialization case - it seems
extremely unlikely, but the fact that entries is NULL doesn't protect
us against a subsequent InstrPushStack running after an OOM and the
check whether InstrStackGrow should be called runs based on the stack
space, not the entries array.

I have a fix for that staged for the next iteration.

> + * 2) Accumulate all instrumentation to the currently active instrumenta=
tion,
> + *    so that callers get a complete picture of activity, even after an =
abort
> ...
> + if (idx >=3D 0)
> + {
> + while (instr_stack.stack_size > idx + 1)
> + instr_stack.stack_size--;
> +
> + InstrPopStack(instr);
> + }
>
> There seems to be one more bug in this:
>
> 1. EXPLAIN ANALYZE fires a trigger
> 2. The trigger function throws ERROR, InstrStopTrigger never runs
> 3. ResOwnerReleaseInstrumentation runs but only checks
> unfinalized_children, not triggers
> 4. InstrStopFinalize discards the trigger entry
> 5. Trigger instrumentation information shows 0

Hmm, so I think you're correct that a trigger function error would
cause any stack-based instrumentation from the trigger to get lost.

In practice that doesn't matter today, since triggers never capture
WAL/buffer usage data (only timing), but its maybe a design flaw
because trigger instrumentation is its own thing without a defined
relationship with the stack, unlike NodeInstrumentation which is
registered to the query that handles the aborts.

In a sense this is a similar situation to the EXPLAIN (SERIALIZE)
per-tuple handling we talked about previously - we have
instrumentation that's related to a query, but its not per-node, so
using NodeInstrumentation doesn't really make sense.

I could imagine three ways forward, if we want to address that now (vs
documenting that this isn't handled but effectively not a problem):

1) We add a second list for unfinalized trigger instrumentations on
QueryInstrumentation, and accumulate that too in
ResOwnerReleaseInstrumentation

2) We call InstrStopFinalize in ExecCallTriggerFunc if an error was
thrown from the trigger function (i.e. turn the existing PG_FINALLY
block into a PG_CATCH)

3) We generalize the QueryInstrumentation children handling to allow
other types of Instrumentation (i.e. not just NodeInstrumentation)

I think (3) would be ideal and would let us deal with EXPLAIN
(SERIALIZE) too, but is complicated by the fact that
ResOwnerReleaseInstrumentation needs to have reference to the full
allocated struct (not just the Instrumentation contained within) so it
can call pfree to avoid leaking memory until top transaction end.

In a prior iteration we had the Instrumentation allocated separately
inside NodeInstrumentation (so its a pointer and can thus be freed
independently / replaced with a copy on clean exit), which allows
ResOwnerReleaseInstrumentation to just deal with Instrumentation, but
that becomes inconvenient when dealing with parallel workers.

There is an alternate design I had considered, which is to basically
keep two copies of Instrumentation in NodeInstrumentation: One is a
pointer to the running instrumentation (allocated in a memory context
that survives long enough in an abort, and which will be freed upon
abort or clean exit), and one is a direct member of the containing
struct (like we have it today), and gets updated via a memcpy() upon a
clean exit. I think that'd make the API easier to use and the same
concept could then be applied to TriggerInstrumentation, but the big
downside is that we'd be doubling memory usage because whilst we're
running we'd both have the allocation in the higher memory context,
and the direct member of the containing struct (to return the result
to the caller).

Because of that, I feel like we should do (1) or (2) for now - but
I'll also wait if Andres or others have additional feedback on 0005
before proceeding with further changes.

I also do think that the 0001-0004 patches are good to be committed
unless anyone had additional feedback there.

Thanks,
Lukas

--
Lukas Fittl