Message-ID: <a1edb578-8a54-4f7a-ad74-11ce9cef291a@iki.fi>
Date: Wed, 25 Mar 2026 12:47:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Stack-based tracking of per-node WAL/buffer usage
To: Lukas Fittl <lukas@fittl.com>, Zsolt Parragi <zsolt.parragi@percona.com>,
 Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tomas@vondra.me>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Peter Smith <smithpb2250@gmail.com>
References: 
 <CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg@mail.gmail.com>
 <CAP53PkxAS-rpp0kcOjg--q5NVNnRC+0kh1+gB8gnRNqacTqF5A@mail.gmail.com>
 <CAN4CZFNuxMqnmtujzemZ1i7hzYkaLD+BWPNq8tFXoPWp5R4How@mail.gmail.com>
 <CAP53PkxFP7i7-wy98ZmEJ11edYq-RrPvJoa4kzGhBBjERA4Nyw@mail.gmail.com>
 <06086cb4-881b-4f5a-96af-f275220ff52d@vondra.me>
 <CAN4CZFOdYhRfecTJjJZCwHtS-gzpSJ16YLQ7Wh0bXrg0=8keOw@mail.gmail.com>
 <CAP53PkxfVnQpcjrwegamPKdyy3RExXXeXNQW1ZWWtr=JtNsLNQ@mail.gmail.com>
 <CAN4CZFMk10YCixVTnt+uBaUH_BH1R=Er-PnPs2YDBhFjuE3K-Q@mail.gmail.com>
 <CAP53Pkwdg7Pmf+OqUBu4_SKgxEH3oosSrxxMc4v=mCqMUB049A@mail.gmail.com>
 <CAP53PkyEPa=aFZrd0crLgRJm1KGS0ykJ+1i2CwDfDNFKpCc+XQ@mail.gmail.com>
 <da7ad516-37d1-47e5-afae-26a7ef7b60f8@iki.fi>
 <CAN4CZFOUTHdXuEAwrST8ueDxGJRe69zVgeVAY0osTjVkRZi_Lw@mail.gmail.com>
 <CAP53Pkx+HZ2OSGDHmNgSDisXdjGp4KFygaX+GaQcFEuXHRXg=g@mail.gmail.com>
 <CAP53PkznofNg+ii363QQGoje30nhssuSz_hV5U4YANAt-Yr_Yg@mail.gmail.com>
Content-Language: en-US
From: Heikki Linnakangas <hlinnaka@iki.fi>
In-Reply-To: 
 <CAP53PkznofNg+ii363QQGoje30nhssuSz_hV5U4YANAt-Yr_Yg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/a1edb578-8a54-4f7a-ad74-11ce9cef291a%40iki.fi>
Precedence: bulk

On 24/03/2026 08:03, Lukas Fittl wrote:
> Instead I've tried introducing a memory context for instrumentation
> managed as a resource owner, and I am now (for now) convinced that
> this is the right trade-off for the problem at hand.

Yes, that seems better.

This patch could use an overview README file, I'm struggling to 
understand how the this all works. Here's my understanding so far, 
please correct me if I'm wrong:

There are *two* data structures tracking the Instrumentation nodes. The 
patch only talks about a stack, but I think there's also implicitly a 
tree in there.

Tree
----

All Instrumentation nodes are part of a tree. For example, if you have 
two portals open, the tree might look like this:

Session - Query A - NestLoop - Seq Scan A
                              - Seq Scan B

         - Query B - Seq Scan C

When a node is "finalized", its counters are added to its parent.

This tree is a somewhat implicit in the patch. Each QueryInstrumentation 
has a list of child nodes, but only unfinalized ones. Don't we need that 
at the session level too? When a Query is released on abort, its 
counters need to be added to the parent too. If I understand correctly, 
the patch tries to use the stack for that, but it's confusing.

I think it would make the patch more clear to talk explicitly about the 
tree, and represent it explicitly in the Instrumentation nodes. I.e. add 
a "parent" pointer, or a "children" list, or both to the Instrumentation 
struct.


Stack
-----

At all times, there's a stack that tracks what is the Instrumentation in 
the tree that is *currently* executing. For example, while executing the 
Seq Scan B, the stack would look like this:

0: Session
1: Query A
2: NestLoop
3: Seq Scan B

And when the code is sending a result row back to the client, while the 
query is being executed, the stack would be just:

0: Session


In the patch, the stack is represented by an array. It could also be 
implemented with a CurrentInstrumentation global variable, similar to 
CurrentMemoryContext and CurrentResourceOwner.


Abort handling
--------------

On abort, two things need to happen:

1. Reset the stack to the appropriate level. This ensures that any we 
don't later try to update the counters on an Instrumentation nodes that 
is going away with the abort. In the above example, the stack would be 
reset to the 0: Session level.

2. Finalize all the Instrumentation nodes as part of the ResourceOwner 
cleanup. All Instrumentation nodes that are released roll up their 
counters to their parents.


Questions:

Is the stack always a path from the root of the tree, down to some node? 
Or could you have e.g. recursion like A -> B -> C -> A? (I don't know if 
it makes a difference, just wondering)

What happens if you release e.g. the NestLoop before its children? All 
the Instrumentation nodes belonging to a query would usually be part of 
the same ResourceOwner and there's no guarantee on what order the 
resources are released.

- Heikki