Date: Wed, 15 Sep 2021 14:40:19 -0700
From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Julien Rouhaud <rjuju123@gmail.com>, Jim Mlodgenski <jimmy76@gmail.com>,
	Simon Riggs <simon.riggs@enterprisedb.com>,
	PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Hook for extensible parsing.
Message-ID: <20210915214019.xtphp3vd2345dir4@alap3.anarazel.de>
References: <20210501072458.adqjoaqnmhg4l34l@nol>
 <CANbhV-EFq08gH_1ZAfenRucq4iR=-2G29Z4f6OPpciy_FqxjiA@mail.gmail.com>
 <CAB_5SRfOGwEhsz6kR4BxYHjjDeUcmCHZRrCtvWQKi+6GL7M7aQ@mail.gmail.com>
 <984300.1631719616@sss.pgh.pa.us>
 <CAOBaU_bpnN7v+yCrK0gt1pxLESNGABV90V2Zd5euFW0TR8hnpQ@mail.gmail.com>
 <1023941.1631725020@sss.pgh.pa.us>
 <20210915195506.bse4pzi2x5atxzzw@alap3.anarazel.de>
 <1091268.1631739097@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1091268.1631739097@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/20210915214019.xtphp3vd2345dir4%40alap3.anarazel.de>
Precedence: bulk

Hi,

On 2021-09-15 16:51:37 -0400, Tom Lane wrote:
> The other problem here is that a simple call-this-instead-of-that
> top-level hook doesn't seem all that useful anyway, because it leaves
> you with the task of duplicating a huge amount of functionality that
> you're then going to make some tweaks within.  That's already an issue
> when you're just thinking about the grammar, and if you have to buy
> into it for parse analysis too, I doubt that it's going to be very
> practical.  If, say, you'd like to support some weird function that
> requires special parsing and analysis rules, I don't see how you get
> that out of this without first duplicating a very large fraction of
> src/backend/parser/.

We do have a small amount of infrastructure around this - the hackery that
plpgsql uses. That's not going to help you with everything, but I think it
should be be enough to recognize e.g. additional top-level
statements. Obviously not enough to intercept parsing deeper into a statement,
but at least something.

And parse-analysis for some types of things will be doable with the current
infrastructure, by e.g. handling the new top-level statement in the hook, and
then passing the buck to the normal parse analysis for e.g. expressions in
that.

Obviously not going to get you that far...


> (As a comparison point, we do have a top-level hook for replacing
> the planner; but I have never heard of anyone actually doing so.
> There are people using that hook to *wrap* the planner with some
> before-and-after processing, which is quite a different thing.)

Citus IIRC has some paths that do not end up calling into the standard
planner, but only for a few simplistic cases.


> I don't have any better ideas to offer :-( ... but I very much fear
> that the approach proposed here is a dead end.

I unfortunately don't see a good way forward without changing the way we do
parsing on a more fundamental level :(.

Greetings,

Andres Freund