From: Tom Lane <tgl@sss.pgh.pa.us>
To: Noah Misch <noah@leadboat.com>
cc: Francesco Degrassi <francesco.degrassi@optionfactory.net>,
        pgsql-bugs@lists.postgresql.org
Subject: Re: Leader backend hang on IPC/ParallelFinish when LWLock held at
 parallel query start
In-reply-to: <20240918050434.13.nmisch@google.com>
References: 
 <CAC-SaSzHUKT=vZJ8MPxYdC_URPfax+yoA1hKTcF4ROz_Q6z0_Q@mail.gmail.com>
 <20240918030159.2a.nmisch@google.com> <1737570.1726633422@sss.pgh.pa.us>
 <20240918050434.13.nmisch@google.com>
Comments: In-reply-to Noah Misch <noah@leadboat.com>
	message dated "Tue, 17 Sep 2024 22:04:34 -0700"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1744647.1726636728.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Wed, 18 Sep 2024 01:18:48 -0400
Message-ID: <1744648.1726636728@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/1744648.1726636728%40sss.pgh.pa.us>
Precedence: bulk

Noah Misch <noah@leadboat.com> writes:
> On Wed, Sep 18, 2024 at 12:23:42AM -0400, Tom Lane wrote:
>> I actually think that the problem is somewhere upstream of here:
>> what in the world are we doing invoking planning and execution of
>> arbitrary queries in a context where interrupts are held off?

> It would be nice to never run planning or execution with interrupts held=
.  The
> concrete examples so far involve btree operator classes with non-C suppo=
rt
> functions.  How practical would it be to release buffer content locks be=
fore
> running index support functions?

Ugh.  Probably not very.  We have to be able to perform comparison
operations within the scanning of a page.  Even if it could be made
to work correctly to release and re-acquire the buffer lock many
times per page, it sounds awful for performance.

> An alternative would be blocking non-C
> support functions (and instructing C support function authors to not use
> planner/executor).  Non-C support functions have been handy for testing =
if
> nothing else, though.  Do non-bundled modules rely on non-C support func=
tions?

Dunno ... but the OP claimed this is a case that's seen in the
field, so maybe somebody is doing it.  On the whole I don't see
how a btree support function can be considered not to be a low-level
thing, so maybe restricting what it can do is the best answer.
I fear though that the restriction can't simply be to forbid
parallel sub-queries.

On the third hand: you can't implement a btree comparison function
in SQL and simultaneously claim with a straight face that you
expect premium performance.  Could we set things up so that
buffer release/reacquires happen only with non-C support functions?
This still requires that such behavior is safe in the face of
concurrent index activity, which I'm not sure is practical.
(Also, I'm not sure to what extent we'd still be testing what
we wish to with test comparison functions written in SQL.)

			regards, tom lane