Feedback-ID: ie8b149e1:Fastmail
User-Agent: Cyrus-JMAP/3.9.0-alpha0-761-gece9e40c48-fm-20230913.001-gece9e40c
MIME-Version: 1.0
Message-Id: <43bc4f5a-2fdc-46c8-ba4f-77ec5d7485e5@app.fastmail.com>
In-Reply-To: <8068eb19f732169a1a79fdeadf104adf@oss.nttdata.com>
References: <3bb7bacbc5ef9e58e2ece0557adac98b@oss.nttdata.com>
 <03c8dcbb-72d2-d3ad-3ad1-9efe69676529@oss.nttdata.com>
 <20220208.171852.1699766364147189900.horikyota.ntt@gmail.com>
 <ab761e68f3fbcee80cf1ca19b1db85f1@oss.nttdata.com>
 <3e94e9b3-83ef-c328-0c5c-f93aa3224790@oss.nttdata.com>
 <3d121ed5f81cef588bac836b43f5d1f9@oss.nttdata.com>
 <379d69c0b3d058997550f88e73ffddd6@oss.nttdata.com>
 <727712bcaa8d449615820c16d15e452e@oss.nttdata.com>
 <598171663576373@mail.yandex.ru>
 <38f4e19323b264e5975c60010b512b15@oss.nttdata.com>
 <20221206184101.exj2zyhmt7ta5fco@awork3.anarazel.de>
 <c51098ac613f908776e5564a7c7a0e16@oss.nttdata.com>
 <CAAaqYe9euUZD8bkjXTVcD9e4n5c7kzHzcvuCJXt-xds9X4c7Fw@mail.gmail.com>
 <2f466f82de025f745dff9377a368cee4@oss.nttdata.com>
 <CAAaqYe8LXVXQhYy3yT0QOHUymdM=uha0dJ0=BEPzVAx2nG1gsw@mail.gmail.com>
 <788e5db60b6c5de4b73fc9810b635d44@oss.nttdata.com>
 <CAAaqYe9u-ywvRX32iBj1SkQrLf1hqMhR9HFCc0dAk4rGDJpyFw@mail.gmail.com>
 <9c3445ab0f695dbd0a879cf380f313c5@oss.nttdata.com>
 <CAAaqYe-5jKTuuZB5StBwa2YRiqpnN=q9buKADP8A_RPd09f_Ug@mail.gmail.com>
 <CAAaqYe-M63VNMpgq0sEw3+t2KaJyhXfu3uBU7w_jtfbGAm4LoA@mail.gmail.com>
 <d81f73616aa5164a68033bc96662dfbc@oss.nttdata.com>
 <CAAaqYe9QB_CUjXrvNh-cpzW6Hz7S4o5pZnR1U1TujLGptUrS1A@mail.gmail.com>
 <00dac323d6732d59ba2c03be729966dc@oss.nttdata.com>
 <af8b4c4752dcee3ca7284e7c15922d5d@oss.nttdata.com>
 <CAAaqYe9TPyWe9yoOm5Jz_39gSnhCAYXozc698jkyegM1x74Qjg@mail.gmail.com>
 <CAAaqYe9Mse7JC78DgT97sXoB6U+E5OL3zJAYRSsgy-TPP9PXmw@mail.gmail.com>
 <4e1e4e5d022964953e84468fe4e511b0@oss.nttdata.com>
 <CAAaqYe_5p_MkBCgSYX=S7G3ZKi=2=_6bZQFzTgVYyv=1NRikpw@mail.gmail.com>
 <4af7712833d4ce989cf5e658106e42da@oss.nttdata.com>
 <CAAaqYe_hyWLs-ObFQfu54yA-ZOjNCd-L_Z=1TnYjYNqrLuGiXQ@mail.gmail.com>
 <cbe6eb783affb3e9b5068f5cf019ee62@oss.nttdata.com>
 <b1b110ae-61f6-4fd9-9b94-f967db9b53d4@app.fastmail.com>
 <8068eb19f732169a1a79fdeadf104adf@oss.nttdata.com>
Date: Wed, 20 Sep 2023 12:39:13 +0700
From: "Lepikhov Andrei" <a.lepikhov@postgrespro.ru>
To: torikoshia <torikoshia@oss.nttdata.com>
Cc: "James Coleman" <jtc331@gmail.com>,
 "'Andres Freund'" <andres@anarazel.de>, pgsql-hackers@lists.postgresql.org,
 "Greg Stark" <stark@mit.edu>, "Ronan Dunklau" <ronan.dunklau@aiven.io>,
 david.christensen@crunchydata.com, "Heikki Linnakangas" <hlinnaka@iki.fi>
Subject: Re: RFC: Logging plan of the running query
Content-Type: text/plain
Archived-At: <https://www.postgresql.org/message-id/43bc4f5a-2fdc-46c8-ba4f-77ec5d7485e5%40app.fastmail.com>
Precedence: bulk

On Tue, Sep 19, 2023, at 8:39 PM, torikoshia wrote:
> On 2023-09-15 15:21, Lepikhov Andrei wrote:
>> On Thu, Sep 7, 2023, at 1:09 PM, torikoshia wrote:
>> I have explored this patch and, by and large, agree with the code. But
>> some questions I want to discuss:
>> 1. Maybe add a hook to give a chance for extensions, like
>> pg_query_state, to do their job?
>
> Do you imagine adding a hook which enables adding custom interrupt codes 
> like below?
>
> https://github.com/postgrespro/pg_query_state/blob/master/patches/custom_signals_15.0.patch

No, I think around the hook, which would allow us to rewrite the  pg_query_state extension without additional patches by using the functionality provided by your patch. I mean, an extension could provide console UI, not only server logging. And obtain from target backend so much information in the explain as the instrumentation level of the current query can give.
It may make pg_query_state shorter and more stable.

>> 2. In this implementation, ProcessInterrupts does a lot of work during
>> the explain creation: memory allocations, pass through the plan,
>> systables locks, syscache access, etc. I guess it can change the
>> semantic meaning of 'safe place' where CHECK_FOR_INTERRUPTS can be
>> called - I can imagine a SELECT query, which would call a stored
>> procedure, which executes some DDL and acquires row exclusive lock at
>> the time of interruption. So, in my mind, it is too unpredictable to
>> make the explain in the place of interruption processing. Maybe it is
>> better to think about some hook at the end of ExecProcNode, where a
>> pending explain could be created?
>
> Yeah, I withdrew this patch once for that reason, but we are resuming 
> development in response to the results of a discussion by James and 
> others at this year's pgcon about the safety of this process in CFI:
>
> https://www.postgresql.org/message-id/CAAaqYe9euUZD8bkjXTVcD9e4n5c7kzHzcvuCJXt-xds9X4c7Fw%40mail.gmail.com

I can't track the logic path  of the decision provided at this conference. But my anxiety related to specific place, where ActiveQueryDesc points top-level query and interruption comes during DDL, wrapped up in stored procedure. For example:
CREATE TABLE test();
CREATE OR REPLACE FUNCTION ddl() RETURNS void AS $$
BEGIN
  EXECUTE format('ALTER TABLE test ADD COLUMN x integer;');
  ...
END; $$ LANGUAGE plpgsql VOLATILE;
SELECT ddl(), ... FROM ...;

> BTW it seems pg_query_state also enables users to get running query 
> plans using CFI.
> Does pg_query_state do something for this safety concern?

No, and I'm looking for the solution, which could help to rewrite pg_query_state as a clean extension, without patches.

>> Also, I suggest minor code change with the diff in attachment.
>
> Thanks!
>
> This might be biased opinion and objections are welcomed, but I feel the 
> original patch is easier to read since PG_RETURN_BOOL(true/false) is 
> located in near place to each cases.
> Also the existing function pg_log_backend_memory_contexts(), which does 
> the same thing, has the same form as the original patch.
I got it, thank you.

-- 
Regards,
Andrei Lepikhov