pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedPG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
75+ messages / 12 participants
[nested] [flat]
Commits from this thread:
2026-07-09 86ab7f4c72 JSON_TABLE PLAN clause
2024-04-08 bb766cde63 JSON_TABLE: Add support for NESTED paths and columns
2024-04-04 de3600452b Add basic JSON_TABLE() functionality
2024-03-21 6185c9737c Add SQL/JSON query functions
2024-01-24 aaaf9449ec Add soft error handling to some expression nodes
2024-01-24 1edb3b491b Adjust populate_record_field() to handle errors softly
2024-01-24 faa2b953ba Refactor code used by jsonpath executor to fetch variables
2023-07-26 03734a7fed Add more SQL/JSON constructor functions
2023-03-31 6ee30209a6 SQL/JSON: support the IS JSON predicate
2023-03-29 7081ac46ac SQL/JSON: add standard JSON constructor functions
2023-02-22 78be04e4c6 Add static assertion ensuring sizeof(ExprEvalStep) <= 64 bytes
2022-07-06 fe3caa1439 30efc3b5a3 Remove size increase in ExprEvalStep caused by hashed saops
2022-07-05 056cc366fa 4a37527fde pgstat: reduce timer overhead by leaving timer running.
2022-07-05 67b26703b4 5a1ab894f7 expression eval: Fix EEOP_JSON_CONSTRUCTOR and EEOP_JSONEXPR size.
* PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
@ 2022-06-16 23:31 Andres Freund <andres@anarazel.de>
2022-06-16 23:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 00:16 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
0 siblings, 3 replies; 75+ messages in thread
From: Andres Freund @ 2022-06-16 23:31 UTC (permalink / raw)
To: pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
Hi,
Mark Callaghan reported a regression in 15, in the post linked here (with
comments in the twitter thread, hence linked here)
https://twitter.com/MarkCallaghanDB/status/1537475430227161098
A differential flame graph shows increased time spent doing memory
allocations, below ExecInitExpr():
https://mdcallag.github.io/reports/22_06_06_ibench.20m.pg.all/l.i0.0.143.15b1.n.svg
Quite the useful comparison.
That lead to me to look at the size of ExprEvalStep - and indeed, it has
*exploded* in 15. 10-13: 64 bytes, 14: 320 bytes.
The comment above the union for data fields says:
/*
* Inline data for the operation. Inline data is faster to access, but
* also bloats the size of all instructions. The union should be kept to
* no more than 40 bytes on 64-bit systems (so that the entire struct is
* no more than 64 bytes, a single cacheline on common systems).
*/
However, jsonexpr/EEOP_JSONEXPR is 296 bytes, and
hashedscalararrayop/EEOP_HASHED_SCALARARRAYOP is 64 bytes, even though the
limit is 40 bytes.
The EEOP_JSONEXPR stuff was added during 15 development in:
commit 1a36bc9dba8eae90963a586d37b6457b32b2fed4
Author: Andrew Dunstan <andrew@dunslane.net>
Date: 2022-03-03 13:11:14 -0500
SQL/JSON query functions
the EEOP_HASHED_SCALARARRAYOP stuff was added during 14 development in:
commit 50e17ad281b8d1c1b410c9833955bc80fbad4078
Author: David Rowley <drowley@postgresql.org>
Date: 2021-04-08 23:51:22 +1200
Speedup ScalarArrayOpExpr evaluation
Unfortunately ExprEvalStep is public, so I don't think we can fix the 14
regression. If somebody installed updated server packages while the server is
running, we could end up loading extensions referencing ExprEvalStep (at least
plpgsql and LLVMJIT).
It's not great to have an ABI break at this point of the 15 cycle, but I don't
think we have a choice. Exploding the size of ExprEvalStep by ~4x is bad -
both for memory overhead (expressions often have dozens to hundreds of steps)
and expression evaluation performance.
The Hashed SAO case can perhaps be squeezed sufficiently to fit inline, but
clearly that's not going to happen for the json case. So we should just move
that out of line.
Maybe it's worth sticking a StaticAssert() for the struct size somewhere. I'm
a bit wary about that being too noisy, there are some machines with odd
alignment requirements. Perhaps worth restricting the assertion to x86-64 +
armv8 or such?
It very well might be that this isn't the full explanation of the regression
Mark observed. E.g. the difference in DecodeDateTime() looks more likely to be
caused by 591e088dd5b - but we need to fix the above issue, whether it's the
cause of the regression Mark observed or not.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-16 23:37 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 21:34 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 1 reply; 75+ messages in thread
From: Tom Lane @ 2022-06-16 23:37 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
Andres Freund <andres@anarazel.de> writes:
> However, jsonexpr/EEOP_JSONEXPR is 296 bytes, and
> hashedscalararrayop/EEOP_HASHED_SCALARARRAYOP is 64 bytes, even though the
> limit is 40 bytes.
Oops.
> Maybe it's worth sticking a StaticAssert() for the struct size
> somewhere.
Indeed. I thought we had one already.
> I'm a bit wary about that being too noisy, there are some machines with
> odd alignment requirements. Perhaps worth restricting the assertion to
> x86-64 + armv8 or such?
I'd put it in first and only reconsider if it shows unfixable problems.
regards, tom lane
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-16 23:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
@ 2023-02-22 21:34 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 22:47 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 75+ messages in thread
From: Tom Lane @ 2023-02-22 21:34 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
I wrote:
> Andres Freund <andres@anarazel.de> writes:
>> Maybe it's worth sticking a StaticAssert() for the struct size
>> somewhere.
> Indeed. I thought we had one already.
>> I'm a bit wary about that being too noisy, there are some machines with
>> odd alignment requirements. Perhaps worth restricting the assertion to
>> x86-64 + armv8 or such?
> I'd put it in first and only reconsider if it shows unfixable problems.
Now that we've got the sizeof(ExprEvalStep) under control, shouldn't
we do the attached?
regards, tom lane
Attachments:
check-sizeof-ExprEvalStep.patch (text/x-diff, 506B) download
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 86e1ac1e65..06c3adc0a1 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -669,6 +669,10 @@ typedef struct ExprEvalStep
} d;
} ExprEvalStep;
+/* Enforce the size rule given in the comment above */
+StaticAssertDecl(sizeof(ExprEvalStep) <= 64,
+ "size of ExprEvalStep exceeds 64 bytes");
+
/* Non-inline data for container operations */
typedef struct SubscriptingRefState
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-16 23:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 21:34 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
@ 2023-02-22 22:47 ` Andres Freund <andres@anarazel.de>
2023-02-23 18:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Corey Huinker <corey.huinker@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2023-02-22 22:47 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
Hi,
On 2023-02-22 16:34:44 -0500, Tom Lane wrote:
> I wrote:
> > Andres Freund <andres@anarazel.de> writes:
> >> Maybe it's worth sticking a StaticAssert() for the struct size
> >> somewhere.
>
> > Indeed. I thought we had one already.
>
> >> I'm a bit wary about that being too noisy, there are some machines with
> >> odd alignment requirements. Perhaps worth restricting the assertion to
> >> x86-64 + armv8 or such?
>
> > I'd put it in first and only reconsider if it shows unfixable problems.
>
> Now that we've got the sizeof(ExprEvalStep) under control, shouldn't
> we do the attached?
Indeed. Pushed.
Let's hope there's no rarely used architecture with odd alignment rules.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-16 23:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 21:34 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 22:47 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2023-02-23 18:39 ` Corey Huinker <corey.huinker@gmail.com>
2023-02-23 18:56 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-23 19:35 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 75+ messages in thread
From: Corey Huinker @ 2023-02-23 18:39 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
On Wed, Feb 22, 2023 at 5:47 PM Andres Freund <andres@anarazel.de> wrote:
> Hi,
>
> On 2023-02-22 16:34:44 -0500, Tom Lane wrote:
> > I wrote:
> > > Andres Freund <andres@anarazel.de> writes:
> > >> Maybe it's worth sticking a StaticAssert() for the struct size
> > >> somewhere.
> >
> > > Indeed. I thought we had one already.
> >
> > >> I'm a bit wary about that being too noisy, there are some machines
> with
> > >> odd alignment requirements. Perhaps worth restricting the assertion to
> > >> x86-64 + armv8 or such?
> >
> > > I'd put it in first and only reconsider if it shows unfixable problems.
> >
> > Now that we've got the sizeof(ExprEvalStep) under control, shouldn't
> > we do the attached?
>
> Indeed. Pushed.
>
> Let's hope there's no rarely used architecture with odd alignment rules.
>
> Greetings,
>
> Andres Freund
>
>
>
I have a question about this that may affect some of my future work.
My not-ready-for-16 work on CAST( ... ON DEFAULT ... ) involved making
FuncExpr/IoCoerceExpr/ArrayCoerceExpr have a safe_mode flag, and that
necessitates adding a reserror boolean to ExprEvalStep for subsequent steps
to test if the error happened.
Will that change be throwing some architectures over the 64 byte count?
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-16 23:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 21:34 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 22:47 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2023-02-23 18:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Corey Huinker <corey.huinker@gmail.com>
@ 2023-02-23 18:56 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-02-23 19:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 75+ messages in thread
From: Tom Lane @ 2023-02-23 18:56 UTC (permalink / raw)
To: Corey Huinker <corey.huinker@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
Corey Huinker <corey.huinker@gmail.com> writes:
> My not-ready-for-16 work on CAST( ... ON DEFAULT ... ) involved making
> FuncExpr/IoCoerceExpr/ArrayCoerceExpr have a safe_mode flag, and that
> necessitates adding a reserror boolean to ExprEvalStep for subsequent steps
> to test if the error happened.
Why do you want it in ExprEvalStep ... couldn't it be in ExprState?
I can't see why you'd need more than one at a time during evaluation.
regards, tom lane
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-16 23:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 21:34 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 22:47 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2023-02-23 18:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Corey Huinker <corey.huinker@gmail.com>
2023-02-23 18:56 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
@ 2023-02-23 19:39 ` Andres Freund <andres@anarazel.de>
2023-02-24 09:44 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Corey Huinker <corey.huinker@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2023-02-23 19:39 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Corey Huinker <corey.huinker@gmail.com>; pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
Hi,
On 2023-02-23 13:56:56 -0500, Tom Lane wrote:
> Corey Huinker <corey.huinker@gmail.com> writes:
> > My not-ready-for-16 work on CAST( ... ON DEFAULT ... ) involved making
> > FuncExpr/IoCoerceExpr/ArrayCoerceExpr have a safe_mode flag, and that
> > necessitates adding a reserror boolean to ExprEvalStep for subsequent steps
> > to test if the error happened.
>
> Why do you want it in ExprEvalStep ... couldn't it be in ExprState?
> I can't see why you'd need more than one at a time during evaluation.
I don't know exactly what CAST( ... ON DEFAULT ... ) is aiming for - I guess
it wants to assign a different value when the cast fails? Is the default
expression a constant, or does it need to be runtime evaluated? If a const,
then the cast steps just could assign the new value. If runtime evaluation is
needed I'd expect the various coerce steps to jump to the value implementing
the default expression in case of a failure.
So I'm not sure we even need a reserror field in ExprState.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-16 23:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 21:34 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 22:47 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2023-02-23 18:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Corey Huinker <corey.huinker@gmail.com>
2023-02-23 18:56 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-23 19:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2023-02-24 09:44 ` Corey Huinker <corey.huinker@gmail.com>
0 siblings, 0 replies; 75+ messages in thread
From: Corey Huinker @ 2023-02-24 09:44 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
On Thu, Feb 23, 2023 at 2:39 PM Andres Freund <andres@anarazel.de> wrote:
> Hi,
>
> On 2023-02-23 13:56:56 -0500, Tom Lane wrote:
> > Corey Huinker <corey.huinker@gmail.com> writes:
> > > My not-ready-for-16 work on CAST( ... ON DEFAULT ... ) involved making
> > > FuncExpr/IoCoerceExpr/ArrayCoerceExpr have a safe_mode flag, and that
> > > necessitates adding a reserror boolean to ExprEvalStep for subsequent
> steps
> > > to test if the error happened.
> >
> > Why do you want it in ExprEvalStep ... couldn't it be in ExprState?
> > I can't see why you'd need more than one at a time during evaluation.
>
> I don't know exactly what CAST( ... ON DEFAULT ... ) is aiming for - I
> guess
> it wants to assign a different value when the cast fails? Is the default
> expression a constant, or does it need to be runtime evaluated? If a
> const,
> then the cast steps just could assign the new value. If runtime evaluation
> is
> needed I'd expect the various coerce steps to jump to the value
> implementing
> the default expression in case of a failure.
>
The default expression is itself a cast expression. So CAST (expr1 AS
some_type DEFAULT expr2 ON ERROR) would basically be a safe-mode cast of
expr1 to some_type, and only upon failure would the non-safe cast of expr2
to some_type be executed. Granted, the most common use case would be for
expr2 to be a constant or something that folds into a constant, but the
proposed spec allows for it.
My implementation involved adding a setting to CoalesceExpr that tested for
error flags rather than null flags, hence putting it in ExprEvalStep and
ExprState (perhaps mistakenly). Copying and adapting EEOP_JUMP_IF_NOT_NULL
lead me to this:
EEO_CASE(EEOP_JUMP_IF_NOT_ERROR)
{
/* Transfer control if current result is non-error */
if (!*op->reserror)
{
*op->reserror = false;
EEO_JUMP(op->d.jump.jumpdone);
}
/* reset error flag */
*op->reserror = false;
EEO_NEXT();
}
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-16 23:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 21:34 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 22:47 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2023-02-23 18:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Corey Huinker <corey.huinker@gmail.com>
@ 2023-02-23 19:35 ` Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 75+ messages in thread
From: Andres Freund @ 2023-02-23 19:35 UTC (permalink / raw)
To: Corey Huinker <corey.huinker@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
Hi,
On 2023-02-23 13:39:14 -0500, Corey Huinker wrote:
> My not-ready-for-16 work on CAST( ... ON DEFAULT ... ) involved making
> FuncExpr/IoCoerceExpr/ArrayCoerceExpr have a safe_mode flag, and that
> necessitates adding a reserror boolean to ExprEvalStep for subsequent steps
> to test if the error happened.
I think if that requires adding a new variable to each ExprEvalStep, it's
DOA. The overhead would be too high. But I don't see why it would need to be
added all ExprEvalSteps instead of individual steps, or perhaps ExprEvalState?
> Will that change be throwing some architectures over the 64 byte count?
It would.
I find the 'pahole' tool very useful for looking at struct layout.
struct ExprEvalStep {
intptr_t opcode; /* 0 8 */
Datum * resvalue; /* 8 8 */
_Bool * resnull; /* 16 8 */
union {
struct {
int last_var; /* 24 4 */
_Bool fixed; /* 28 1 */
/* XXX 3 bytes hole, try to pack */
TupleDesc known_desc; /* 32 8 */
const TupleTableSlotOps * kind; /* 40 8 */
} fetch; /* 24 24 */
...
struct {
Datum * values; /* 24 8 */
_Bool * nulls; /* 32 8 */
int nelems; /* 40 4 */
MinMaxOp op; /* 44 4 */
FmgrInfo * finfo; /* 48 8 */
FunctionCallInfo fcinfo_data; /* 56 8 */
} minmax; /* 24 40 */
...
} d; /* 24 40 */
/* size: 64, cachelines: 1, members: 4 */
};
We don't have memory to spare in the "general" portion of ExprEvalStep
(currently 24 bytes), as several of the type-specific portions are already 40
bytes large.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-17 00:16 ` Andres Freund <andres@anarazel.de>
2 siblings, 0 replies; 75+ messages in thread
From: Andres Freund @ 2022-06-17 00:16 UTC (permalink / raw)
To: pgsql-hackers@postgresql.org, Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>
Hi,
On 2022-06-16 16:31:30 -0700, Andres Freund wrote:
> The EEOP_JSONEXPR stuff was added during 15 development in:
>
> commit 1a36bc9dba8eae90963a586d37b6457b32b2fed4
> Author: Andrew Dunstan <andrew@dunslane.net>
> Date: 2022-03-03 13:11:14 -0500
>
> SQL/JSON query functions
I'm quite confused about part of the struct definition of this:
struct JsonCoercionsState
{
struct JsonCoercionState
{
JsonCoercion *coercion; /* coercion expression */
ExprState *estate; /* coercion expression state */
} null,
string,
numeric ,
boolean,
date,
time,
timetz,
timestamp,
timestamptz,
composite;
} coercions; /* states for coercion from SQL/JSON item
* types directly to the output type */
Why on earth do we have coercion state for all these different types? That
really adds up:
struct {
JsonExpr * jsexpr; /* 24 8 */
struct {
FmgrInfo func; /* 32 48 */
/* --- cacheline 1 boundary (64 bytes) was 16 bytes ago --- */
Oid typioparam; /* 80 4 */
} input; /* 32 56 */
/* XXX last struct has 4 bytes of padding */
NullableDatum * formatted_expr; /* 88 8 */
NullableDatum * res_expr; /* 96 8 */
NullableDatum * coercion_expr; /* 104 8 */
NullableDatum * pathspec; /* 112 8 */
ExprState * result_expr; /* 120 8 */
/* --- cacheline 2 boundary (128 bytes) --- */
ExprState * default_on_empty; /* 128 8 */
ExprState * default_on_error; /* 136 8 */
List * args; /* 144 8 */
void * cache; /* 152 8 */
struct JsonCoercionsState coercions; /* 160 160 */
} jsonexpr; /* 24 296 */
And why is FmgrInfo stored inline in the struct? Everything else just stores
pointers to FmgrInfo.
Now that I look at this: It's a *bad* idea to have subsidiary ExprState inside
an ExprState. Nearly always the correct thing to do is to build those
expressions. There's plenty memory and evaluation overhead in jumping to a
different expression. And I see no reason for doing it that way here?
This stuff doesn't look ready.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-17 02:14 ` David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 12:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size James Coleman <jtc331@gmail.com>
2022-06-17 17:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2 siblings, 3 replies; 75+ messages in thread
From: David Rowley @ 2022-06-17 02:14 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
On Fri, 17 Jun 2022 at 11:31, Andres Freund <andres@anarazel.de> wrote:
> hashedscalararrayop/EEOP_HASHED_SCALARARRAYOP is 64 bytes, even though the
> limit is 40 bytes.
> commit 50e17ad281b8d1c1b410c9833955bc80fbad4078
> Author: David Rowley <drowley@postgresql.org>
> Date: 2021-04-08 23:51:22 +1200
>
> Speedup ScalarArrayOpExpr evaluation
I've put together the attached patch which removes 4 fields from the
hashedscalararrayop portion of the struct which, once the JSON part is
fixed, will put sizeof(ExprEvalStep) back down to 64 bytes again.
The attached patch causes some extra pointer dereferencing to perform
a hashed saop step, so I tested the performance on f4fb45d15 (prior to
the JSON patch that pushed the sizeof(ExprEvalStep) up further. I
found:
setup:
create table a (a int);
insert into a select x from generate_series(1000000,2000000) x;
bench.sql
select * from a where a in(1,2,3,4,5,6,7,8,9,10);
f4fb45d15 + reduce_sizeof_hashedsaop_ExprEvalStep.patch
drowley@amd3990x:~$ pgbench -n -f bench.sql -T 60 -M prepared postgres
tps = 44.841851 (without initial connection time)
tps = 44.986472 (without initial connection time)
tps = 44.944315 (without initial connection time)
f4fb45d15
drowley@amd3990x:~$ pgbench -n -f bench.sql -T 60 -M prepared postgres
tps = 44.446127 (without initial connection time)
tps = 44.614134 (without initial connection time)
tps = 44.895011 (without initial connection time)
(Patched is ~0.61% faster here)
So, there appears to be no performance regression due to the extra
indirection. There's maybe even some gains due to the smaller step
size.
David
Attachments:
reduce_sizeof_hashedsaop_ExprEvalStep.patch (text/plain, 3.9K) download
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 2831e7978b..82bf47ddb9 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -1270,10 +1270,6 @@ ExecInitExprRec(Expr *node, ExprState *state,
1, opexpr->inputcollid, NULL,
NULL);
- scratch.d.hashedscalararrayop.hash_finfo = hash_finfo;
- scratch.d.hashedscalararrayop.hash_fcinfo_data = hash_fcinfo;
- scratch.d.hashedscalararrayop.hash_fn_addr = hash_finfo->fn_addr;
-
/* Evaluate scalar directly into left function argument */
ExecInitExprRec(scalararg, state,
&fcinfo->args[0].value, &fcinfo->args[0].isnull);
@@ -1290,13 +1286,8 @@ ExecInitExprRec(Expr *node, ExprState *state,
/* And perform the operation */
scratch.opcode = EEOP_HASHED_SCALARARRAYOP;
scratch.d.hashedscalararrayop.inclause = opexpr->useOr;
- scratch.d.hashedscalararrayop.finfo = finfo;
scratch.d.hashedscalararrayop.fcinfo_data = fcinfo;
- scratch.d.hashedscalararrayop.fn_addr = finfo->fn_addr;
-
- scratch.d.hashedscalararrayop.hash_finfo = hash_finfo;
scratch.d.hashedscalararrayop.hash_fcinfo_data = hash_fcinfo;
- scratch.d.hashedscalararrayop.hash_fn_addr = hash_finfo->fn_addr;
ExprEvalPushStep(state, &scratch);
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index eaec697bb3..c8dc859e1d 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3480,7 +3480,7 @@ saop_element_hash(struct saophash_hash *tb, Datum key)
fcinfo->args[0].value = key;
fcinfo->args[0].isnull = false;
- hash = elements_tab->op->d.hashedscalararrayop.hash_fn_addr(fcinfo);
+ hash = elements_tab->op->d.hashedscalararrayop.hash_fcinfo_data->flinfo->fn_addr(fcinfo);
return DatumGetUInt32(hash);
}
@@ -3502,7 +3502,7 @@ saop_hash_element_match(struct saophash_hash *tb, Datum key1, Datum key2)
fcinfo->args[1].value = key2;
fcinfo->args[1].isnull = false;
- result = elements_tab->op->d.hashedscalararrayop.fn_addr(fcinfo);
+ result = elements_tab->op->d.hashedscalararrayop.fcinfo_data->flinfo->fn_addr(fcinfo);
return DatumGetBool(result);
}
@@ -3526,7 +3526,7 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
ScalarArrayOpExprHashTable *elements_tab = op->d.hashedscalararrayop.elements_tab;
FunctionCallInfo fcinfo = op->d.hashedscalararrayop.fcinfo_data;
bool inclause = op->d.hashedscalararrayop.inclause;
- bool strictfunc = op->d.hashedscalararrayop.finfo->fn_strict;
+ bool strictfunc = op->d.hashedscalararrayop.fcinfo_data->flinfo->fn_strict;
Datum scalar = fcinfo->args[0].value;
bool scalar_isnull = fcinfo->args[0].isnull;
Datum result;
@@ -3669,7 +3669,7 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
fcinfo->args[1].value = (Datum) 0;
fcinfo->args[1].isnull = true;
- result = op->d.hashedscalararrayop.fn_addr(fcinfo);
+ result = op->d.hashedscalararrayop.fcinfo_data->flinfo->fn_addr(fcinfo);
resultnull = fcinfo->isnull;
/*
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index e34db8c93c..4bb6d34275 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -580,14 +580,8 @@ typedef struct ExprEvalStep
bool has_nulls;
bool inclause; /* true for IN and false for NOT IN */
struct ScalarArrayOpExprHashTable *elements_tab;
- FmgrInfo *finfo; /* function's lookup data */
FunctionCallInfo fcinfo_data; /* arguments etc */
- /* faster to access without additional indirection: */
- PGFunction fn_addr; /* actual call address */
- FmgrInfo *hash_finfo; /* function's lookup data */
FunctionCallInfo hash_fcinfo_data; /* arguments etc */
- /* faster to access without additional indirection: */
- PGFunction hash_fn_addr; /* actual call address */
} hashedscalararrayop;
/* for EEOP_XMLEXPR */
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
@ 2022-06-17 03:33 ` Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2 siblings, 1 reply; 75+ messages in thread
From: Peter Geoghegan @ 2022-06-17 03:33 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
On Thu, Jun 16, 2022 at 7:15 PM David Rowley <dgrowleyml@gmail.com> wrote:
> So, there appears to be no performance regression due to the extra
> indirection. There's maybe even some gains due to the smaller step
> size.
Have you tried this with the insert benchmark [1]?
I've run it myself in the past (when working on B-Tree deduplication).
It's quite straightforward to set up and run.
[1] http://smalldatum.blogspot.com/2017/06/the-insert-benchmark.html
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
@ 2022-06-17 04:53 ` David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 75+ messages in thread
From: David Rowley @ 2022-06-17 04:53 UTC (permalink / raw)
To: Peter Geoghegan <pg@bowt.ie>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
On Fri, 17 Jun 2022 at 15:33, Peter Geoghegan <pg@bowt.ie> wrote:
> Have you tried this with the insert benchmark [1]?
I was mostly focusing on the performance of the hashed saop feature
after having removed the additional fields that pushed ExprEvalStep
over 64 bytes in 14.
I agree it would be good to do further benchmarking to see if there's
anything else that's snuck into 15 that's slowed that benchmark down,
but we can likely work on that after we get the ExprEvalStep size back
to 64 bytes again.
David
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
@ 2022-06-17 05:22 ` Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-06-17 05:22 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Hi,
On 2022-06-17 16:53:31 +1200, David Rowley wrote:
> On Fri, 17 Jun 2022 at 15:33, Peter Geoghegan <pg@bowt.ie> wrote:
> > Have you tried this with the insert benchmark [1]?
>
> I was mostly focusing on the performance of the hashed saop feature
> after having removed the additional fields that pushed ExprEvalStep
> over 64 bytes in 14.
>
> I agree it would be good to do further benchmarking to see if there's
> anything else that's snuck into 15 that's slowed that benchmark down,
> but we can likely work on that after we get the ExprEvalStep size back
> to 64 bytes again.
I did reproduce a regression between 14 and 15, using both pgbench -Mprepared
-S (scale 1) and TPC-H Q01 (scale 5). Between 7-10% - not good, particularly
that that's not been found so far. Fixing the json size issue gets that down
to ~2%. Not sure what that's caused by yet.
Greetings,
Andres Freund
Attachments: download all (2 files, .tar.gz)
v1-0001-wip-fix-EEOP_HASHED_SCALARARRAYOP-state-width.patch (text/x-diff, 5.5K) download
From 8c1ec2f6dce304ae600177a4b5bdb936e8fb65b0 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Thu, 16 Jun 2022 18:28:39 -0700
Subject: [PATCH v1 1/2] wip: fix EEOP_HASHED_SCALARARRAYOP state width.
Author:
Reviewed-By:
Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de
Backpatch:
---
src/include/executor/execExpr.h | 5 -----
src/backend/executor/execExpr.c | 10 ----------
src/backend/executor/execExprInterp.c | 18 ++++++++++++++----
3 files changed, 14 insertions(+), 19 deletions(-)
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index e34db8c93cb..2fba1450c65 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -582,12 +582,7 @@ typedef struct ExprEvalStep
struct ScalarArrayOpExprHashTable *elements_tab;
FmgrInfo *finfo; /* function's lookup data */
FunctionCallInfo fcinfo_data; /* arguments etc */
- /* faster to access without additional indirection: */
- PGFunction fn_addr; /* actual call address */
FmgrInfo *hash_finfo; /* function's lookup data */
- FunctionCallInfo hash_fcinfo_data; /* arguments etc */
- /* faster to access without additional indirection: */
- PGFunction hash_fn_addr; /* actual call address */
} hashedscalararrayop;
/* for EEOP_XMLEXPR */
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 2831e7978b5..4128248be40 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -1204,7 +1204,6 @@ ExecInitExprRec(Expr *node, ExprState *state,
FunctionCallInfo fcinfo;
AclResult aclresult;
FmgrInfo *hash_finfo;
- FunctionCallInfo hash_fcinfo;
Oid cmpfuncid;
/*
@@ -1263,16 +1262,10 @@ ExecInitExprRec(Expr *node, ExprState *state,
if (OidIsValid(opexpr->hashfuncid))
{
hash_finfo = palloc0(sizeof(FmgrInfo));
- hash_fcinfo = palloc0(SizeForFunctionCallInfo(1));
fmgr_info(opexpr->hashfuncid, hash_finfo);
fmgr_info_set_expr((Node *) node, hash_finfo);
- InitFunctionCallInfoData(*hash_fcinfo, hash_finfo,
- 1, opexpr->inputcollid, NULL,
- NULL);
scratch.d.hashedscalararrayop.hash_finfo = hash_finfo;
- scratch.d.hashedscalararrayop.hash_fcinfo_data = hash_fcinfo;
- scratch.d.hashedscalararrayop.hash_fn_addr = hash_finfo->fn_addr;
/* Evaluate scalar directly into left function argument */
ExecInitExprRec(scalararg, state,
@@ -1292,11 +1285,8 @@ ExecInitExprRec(Expr *node, ExprState *state,
scratch.d.hashedscalararrayop.inclause = opexpr->useOr;
scratch.d.hashedscalararrayop.finfo = finfo;
scratch.d.hashedscalararrayop.fcinfo_data = fcinfo;
- scratch.d.hashedscalararrayop.fn_addr = finfo->fn_addr;
scratch.d.hashedscalararrayop.hash_finfo = hash_finfo;
- scratch.d.hashedscalararrayop.hash_fcinfo_data = hash_fcinfo;
- scratch.d.hashedscalararrayop.hash_fn_addr = hash_finfo->fn_addr;
ExprEvalPushStep(state, &scratch);
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index eaec697bb38..77e1cb1b7b4 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -217,6 +217,7 @@ typedef struct ScalarArrayOpExprHashTable
{
saophash_hash *hashtab; /* underlying hash table */
struct ExprEvalStep *op;
+ FunctionCallInfo hash_fcinfo_data; /* arguments etc */
} ScalarArrayOpExprHashTable;
/* Define parameters for ScalarArrayOpExpr hash table code generation. */
@@ -3474,13 +3475,13 @@ static uint32
saop_element_hash(struct saophash_hash *tb, Datum key)
{
ScalarArrayOpExprHashTable *elements_tab = (ScalarArrayOpExprHashTable *) tb->private_data;
- FunctionCallInfo fcinfo = elements_tab->op->d.hashedscalararrayop.hash_fcinfo_data;
+ FunctionCallInfo fcinfo = elements_tab->hash_fcinfo_data;
Datum hash;
fcinfo->args[0].value = key;
fcinfo->args[0].isnull = false;
- hash = elements_tab->op->d.hashedscalararrayop.hash_fn_addr(fcinfo);
+ hash = elements_tab->op->d.hashedscalararrayop.hash_finfo->fn_addr(fcinfo);
return DatumGetUInt32(hash);
}
@@ -3502,7 +3503,7 @@ saop_hash_element_match(struct saophash_hash *tb, Datum key1, Datum key2)
fcinfo->args[1].value = key2;
fcinfo->args[1].isnull = false;
- result = elements_tab->op->d.hashedscalararrayop.fn_addr(fcinfo);
+ result = elements_tab->op->d.hashedscalararrayop.finfo->fn_addr(fcinfo);
return DatumGetBool(result);
}
@@ -3575,6 +3576,15 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
op->d.hashedscalararrayop.elements_tab = elements_tab;
elements_tab->op = op;
+ elements_tab->hash_fcinfo_data = palloc0(SizeForFunctionCallInfo(1));
+ InitFunctionCallInfoData(*elements_tab->hash_fcinfo_data,
+ op->d.hashedscalararrayop.hash_finfo,
+ 1,
+ /* FIXME */
+ ((OpExpr*)op->d.hashedscalararrayop.hash_finfo->fn_expr)->inputcollid,
+ NULL,
+ NULL);
+
/*
* Create the hash table sizing it according to the number of elements
* in the array. This does assume that the array has no duplicates.
@@ -3669,7 +3679,7 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
fcinfo->args[1].value = (Datum) 0;
fcinfo->args[1].isnull = true;
- result = op->d.hashedscalararrayop.fn_addr(fcinfo);
+ result = op->d.hashedscalararrayop.finfo->fn_addr(fcinfo);
resultnull = fcinfo->isnull;
/*
--
2.35.1.677.gabf474a5dd
v1-0002-WIP-Fix-EEOP_JSON_CONSTRUCTOR-and-EEOP_JSONEXPR-s.patch (text/x-diff, 906B) download
From beb2e9aa2c6fef1eb0f513716db6d6ffedddc39b Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Thu, 16 Jun 2022 18:33:42 -0700
Subject: [PATCH v1 2/2] WIP: Fix EEOP_JSON_CONSTRUCTOR and EEOP_JSONEXPR size.
Author:
Reviewed-By:
Discussion: https://postgr.es/m/
Backpatch:
---
src/include/executor/execExpr.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 2fba1450c65..5bfa7e9bdd4 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -22,6 +22,8 @@ struct ExprEvalStep;
struct SubscriptingRefState;
struct ScalarArrayOpExprHashTable;
struct JsonbValue;
+struct JsonExprState;
+struct JsonConstructorExprState;
/* Bits in ExprState->flags (see also execnodes.h for public flag bits): */
/* expression's interpreter has been initialized */
--
2.35.1.677.gabf474a5dd
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-17 06:24 ` Andres Freund <andres@anarazel.de>
2022-06-17 06:54 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 75+ messages in thread
From: Andres Freund @ 2022-06-17 06:24 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Hi,
On 2022-06-16 22:22:28 -0700, Andres Freund wrote:
> On 2022-06-17 16:53:31 +1200, David Rowley wrote:
> > On Fri, 17 Jun 2022 at 15:33, Peter Geoghegan <pg@bowt.ie> wrote:
> > > Have you tried this with the insert benchmark [1]?
> >
> > I was mostly focusing on the performance of the hashed saop feature
> > after having removed the additional fields that pushed ExprEvalStep
> > over 64 bytes in 14.
> >
> > I agree it would be good to do further benchmarking to see if there's
> > anything else that's snuck into 15 that's slowed that benchmark down,
> > but we can likely work on that after we get the ExprEvalStep size back
> > to 64 bytes again.
>
> I did reproduce a regression between 14 and 15, using both pgbench -Mprepared
> -S (scale 1) and TPC-H Q01 (scale 5). Between 7-10% - not good, particularly
> that that's not been found so far. Fixing the json size issue gets that down
> to ~2%. Not sure what that's caused by yet.
The remaining difference looks like it's largely caused by the
enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT, ...) introduced as part of the
pgstats patch. It's only really visible when I pin a single connection pgbench
to the same CPU core as the server (which gives a ~16% boost here).
It's not the timeout itself - that we amortize nicely (via 09cf1d522). It's
that enable_timeout_after() does a GetCurrentTimestamp().
Not sure yet what the best way to fix that is.
We could just leave the timer active and add some gating condition indicating
idleness to the IdleStatsUpdateTimeoutPending body in ProcessInterrupts()?
Or we could add a timeout.c API that specifies the timeout?
pgstat_report_stat() uses GetCurrentTransactionStopTimestamp(), it seems like
it'd make sense to use the same for arming the timeout?
- Andres
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-17 06:54 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-06-17 06:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Kyotaro Horiguchi <horikyota.ntt@gmail.com>
1 sibling, 1 reply; 75+ messages in thread
From: Kyotaro Horiguchi @ 2022-06-17 06:54 UTC (permalink / raw)
To: andres@anarazel.de; +Cc: dgrowleyml@gmail.com; pg@bowt.ie; pgsql-hackers; n.gluhov@postgrespro.ru; andrew@dunslane.net
At Thu, 16 Jun 2022 23:24:56 -0700, Andres Freund <andres@anarazel.de> wrote in
> The remaining difference looks like it's largely caused by the
> enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT, ...) introduced as part of the
> pgstats patch. It's only really visible when I pin a single connection pgbench
> to the same CPU core as the server (which gives a ~16% boost here).
>
> It's not the timeout itself - that we amortize nicely (via 09cf1d522). It's
> that enable_timeout_after() does a GetCurrentTimestamp().
>
> Not sure yet what the best way to fix that is.
>
> We could just leave the timer active and add some gating condition indicating
> idleness to the IdleStatsUpdateTimeoutPending body in ProcessInterrupts()?
>
> Or we could add a timeout.c API that specifies the timeout?
I sometimes wanted this, But I don't see a simple way to sort multiple
relative timeouts in absolute time order. Maybe we can skip
GetCurrentTimestamp only when inserting the first timeout, but I don't
think it benefits this case.
> pgstat_report_stat() uses GetCurrentTransactionStopTimestamp(), it seems like
> it'd make sense to use the same for arming the timeout?
This seems like the feasible best fix for this specific issue.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:54 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Kyotaro Horiguchi <horikyota.ntt@gmail.com>
@ 2022-06-17 06:59 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-06-17 07:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Kyotaro Horiguchi @ 2022-06-17 06:59 UTC (permalink / raw)
To: andres@anarazel.de; +Cc: dgrowleyml@gmail.com; pg@bowt.ie; pgsql-hackers; n.gluhov@postgrespro.ru; andrew@dunslane.net
At Fri, 17 Jun 2022 15:54:13 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > Or we could add a timeout.c API that specifies the timeout?
>
> I sometimes wanted this, But I don't see a simple way to sort multiple
> relative timeouts in absolute time order. Maybe we can skip
> GetCurrentTimestamp only when inserting the first timeout, but I don't
> think it benefits this case.
Or we can use a free-run interval timer and individual down-counter
for each timtouts. I think we need at-most 0.1s resolution and error
of long-run timer doesn't harm?
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:54 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-06-17 06:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Kyotaro Horiguchi <horikyota.ntt@gmail.com>
@ 2022-06-17 07:05 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 0 replies; 75+ messages in thread
From: Kyotaro Horiguchi @ 2022-06-17 07:05 UTC (permalink / raw)
To: andres@anarazel.de; +Cc: dgrowleyml@gmail.com; pg@bowt.ie; pgsql-hackers; n.gluhov@postgrespro.ru; andrew@dunslane.net
At Fri, 17 Jun 2022 15:59:26 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> At Fri, 17 Jun 2022 15:54:13 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > > Or we could add a timeout.c API that specifies the timeout?
> >
> > I sometimes wanted this, But I don't see a simple way to sort multiple
> > relative timeouts in absolute time order. Maybe we can skip
> > GetCurrentTimestamp only when inserting the first timeout, but I don't
> > think it benefits this case.
>
> Or we can use a free-run interval timer and individual down-counter
> for each timtouts. I think we need at-most 0.1s resolution and error
> of long-run timer doesn't harm?
Yeah, stupid. We don't want awake process with such a high frequency..
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-17 14:33 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 14:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
1 sibling, 2 replies; 75+ messages in thread
From: Tom Lane @ 2022-06-17 14:33 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Andres Freund <andres@anarazel.de> writes:
> The remaining difference looks like it's largely caused by the
> enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT, ...) introduced as part of the
> pgstats patch. It's only really visible when I pin a single connection pgbench
> to the same CPU core as the server (which gives a ~16% boost here).
> It's not the timeout itself - that we amortize nicely (via 09cf1d522). It's
> that enable_timeout_after() does a GetCurrentTimestamp().
> Not sure yet what the best way to fix that is.
Maybe not queue a new timeout if the old one is still active?
BTW, it looks like that patch also falsified this comment
(postgres.c:4478):
* At most one of these timeouts will be active, so there's no need to
* worry about combining the timeout.c calls into one.
Maybe fixing that end of things would be a simpler way of buying back
the delta.
> Or we could add a timeout.c API that specifies the timeout?
Don't think that will help: it'd be morally equivalent to
enable_timeout_at(), which also has to do GetCurrentTimestamp().
regards, tom lane
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-06-17 14:53 ` Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 75+ messages in thread
From: Tom Lane @ 2022-06-17 14:53 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
I wrote:
> Andres Freund <andres@anarazel.de> writes:
>> Or we could add a timeout.c API that specifies the timeout?
> Don't think that will help: it'd be morally equivalent to
> enable_timeout_at(), which also has to do GetCurrentTimestamp().
BTW, if we were willing to drop get_timeout_start_time(), it might
be possible to avoid doing GetCurrentTimestamp() in enable_timeout_at,
in the common case where the specified timestamp is beyond signal_due_at
so that no setitimer call is needed. But getting the race conditions
right could be tricky. On the whole this doesn't sound like something
to tackle post-beta.
regards, tom lane
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-06-17 17:30 ` Andres Freund <andres@anarazel.de>
2022-06-17 17:43 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
1 sibling, 2 replies; 75+ messages in thread
From: Andres Freund @ 2022-06-17 17:30 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Hi,
On 2022-06-17 10:33:08 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > The remaining difference looks like it's largely caused by the
> > enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT, ...) introduced as part of the
> > pgstats patch. It's only really visible when I pin a single connection pgbench
> > to the same CPU core as the server (which gives a ~16% boost here).
>
> > It's not the timeout itself - that we amortize nicely (via 09cf1d522). It's
> > that enable_timeout_after() does a GetCurrentTimestamp().
>
> > Not sure yet what the best way to fix that is.
>
> Maybe not queue a new timeout if the old one is still active?
Right now we disable the timer after ReadCommand(). We can of course change
that. At first I thought we might need more bookkeeping to do so, to avoid
ProcessInterrupts() triggering pgstat_report_stat() when the timer fires
later, but we probably can jury-rig something with DoingCommandRead &&
IsTransactionOrTransactionBlock() or such.
I guess one advantage of something like this could be that we could possibly
move the arming of the timeout to pgstat.c. But that looks like it might be
more complicated than really worth it.
> BTW, it looks like that patch also falsified this comment
> (postgres.c:4478):
>
> * At most one of these timeouts will be active, so there's no need to
> * worry about combining the timeout.c calls into one.
Hm, yea. I guess we can just disable them at once.
> Maybe fixing that end of things would be a simpler way of buying back
> the delta.
I don't think that'll do the trick - in the case I'm looking at none of the
other timers are active...
> > Or we could add a timeout.c API that specifies the timeout?
>
> Don't think that will help: it'd be morally equivalent to
> enable_timeout_at(), which also has to do GetCurrentTimestamp().
I should have been more precise - what I meant was a timeout.c API that allows
the caller to pass in "now", which in this case we'd get from
GetCurrentTransactionStopTimestamp(), which would avoid the additional
timestamp computation.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-17 17:43 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 18:02 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 75+ messages in thread
From: Tom Lane @ 2022-06-17 17:43 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Andres Freund <andres@anarazel.de> writes:
> I should have been more precise - what I meant was a timeout.c API that allows
> the caller to pass in "now", which in this case we'd get from
> GetCurrentTransactionStopTimestamp(), which would avoid the additional
> timestamp computation.
I don't care for that one bit: it makes the accuracy of all timeouts
dependent on how careful that caller is to provide an up-to-date "now".
In the example at hand, there is WAY too much code between
SetCurrentTransactionStopTimestamp() and the timer arming to make me
think the results will be acceptable.
regards, tom lane
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 17:43 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-06-17 18:02 ` Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 75+ messages in thread
From: Andres Freund @ 2022-06-17 18:02 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Hi,
On 2022-06-17 13:43:49 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > I should have been more precise - what I meant was a timeout.c API that allows
> > the caller to pass in "now", which in this case we'd get from
> > GetCurrentTransactionStopTimestamp(), which would avoid the additional
> > timestamp computation.
>
> I don't care for that one bit: it makes the accuracy of all timeouts
> dependent on how careful that caller is to provide an up-to-date "now".
I don't think it'd necessarily have to influence the accuracy of all timeouts
- but I've not looked at timeout.c much before. From what I understand we use
'now' for two things: First, to set ->start_time in enable_timeout() and
second to schedule the alarm in schedule_alarm(). An inaccurate start_time
won't cause problems for other timers afaics and it looks to me that it
wouldn't be too hard to only require an accurate 'now' if the new timeout is
nearest_timeout and now + nearest_timeout < signal_due_at?
It's probably to complicated to tinker with now tho.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-17 20:06 ` Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-28 23:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
1 sibling, 2 replies; 75+ messages in thread
From: Andres Freund @ 2022-06-17 20:06 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Hi,
On 2022-06-17 10:30:55 -0700, Andres Freund wrote:
> On 2022-06-17 10:33:08 -0400, Tom Lane wrote:
> > Andres Freund <andres@anarazel.de> writes:
> > > The remaining difference looks like it's largely caused by the
> > > enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT, ...) introduced as part of the
> > > pgstats patch. It's only really visible when I pin a single connection pgbench
> > > to the same CPU core as the server (which gives a ~16% boost here).
> >
> > > It's not the timeout itself - that we amortize nicely (via 09cf1d522). It's
> > > that enable_timeout_after() does a GetCurrentTimestamp().
> >
> > > Not sure yet what the best way to fix that is.
> >
> > Maybe not queue a new timeout if the old one is still active?
>
> Right now we disable the timer after ReadCommand(). We can of course change
> that. At first I thought we might need more bookkeeping to do so, to avoid
> ProcessInterrupts() triggering pgstat_report_stat() when the timer fires
> later, but we probably can jury-rig something with DoingCommandRead &&
> IsTransactionOrTransactionBlock() or such.
Here's a patch for that.
One thing I noticed is that disable_timeout() calls do
schedule_alarm(GetCurrentTimestamp()) if there's any other active timeout,
even if the to-be-disabled timer is already disabled. Of course callers of
disable_timeout() can guard against that using get_timeout_active(), but that
spreads repetitive code around...
I opted to add a fastpath for that, instead of using
get_timeout_active(). Afaics that's safe to do without disarming the signal
handler, but I'd welcome a look from somebody that knows this code.
> I guess one advantage of something like this could be that we could possibly
> move the arming of the timeout to pgstat.c. But that looks like it might be
> more complicated than really worth it.
I didn't do that yet, but am curious whether others think this would be
preferrable.
> > BTW, it looks like that patch also falsified this comment
> > (postgres.c:4478):
> >
> > * At most one of these timeouts will be active, so there's no need to
> > * worry about combining the timeout.c calls into one.
>
> Hm, yea. I guess we can just disable them at once.
With the proposed change we don't need to change the separate timeout.c to
one, or update the comment, as it should now look the same as 14.
I also attached my heavily-WIP patches for the ExprEvalStep issues, I
accidentally had only included a small part of the contents of the json fix.
Greetings,
Andres Freund
Attachments: download all (4 files, .tar.gz)
v2-0001-Add-already-disabled-fastpath-to-disable_timeout.patch (text/x-diff, 1.2K) download
From 5fc220ce4ed8406e7c6852c6a6c59fb74a6e3f82 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Fri, 17 Jun 2022 12:51:20 -0700
Subject: [PATCH v2 1/4] Add already-disabled fastpath to disable_timeout().
Otherwise callers that might call disable_timeout() frequently need to check
get_timeout_active() to avoid GetCurrentTimestamp() and other overheads in
disable_timeout().
Needed to avoid get_timeout_active() check in the subsequent commit fixing a
small performance regression.
---
src/backend/utils/misc/timeout.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/src/backend/utils/misc/timeout.c b/src/backend/utils/misc/timeout.c
index 6f5e08bc302..d1219afa614 100644
--- a/src/backend/utils/misc/timeout.c
+++ b/src/backend/utils/misc/timeout.c
@@ -692,6 +692,11 @@ disable_timeout(TimeoutId id, bool keep_indicator)
Assert(all_timeouts_initialized);
Assert(all_timeouts[id].timeout_handler != NULL);
+ /* fast path for an already disabled timer */
+ if (!all_timeouts[id].active &&
+ (!all_timeouts[id].indicator || keep_indicator))
+ return;
+
/* Disable timeout interrupts for safety. */
disable_alarm();
--
2.35.1.677.gabf474a5dd
v2-0002-WIP-pgstat-reduce-timer-overhead.patch (text/x-diff, 4.5K) download
From 41ff50e1264d2d32bc35b8b2c6c4ca375c90017a Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Fri, 17 Jun 2022 12:48:34 -0700
Subject: [PATCH v2 2/4] WIP: pgstat: reduce timer overhead.
While at it, also update assertion in pgstat_report_stat() to be more precise.
Author:
Reviewed-By:
Discussion: https://postgr.es/m/
Backpatch:
---
src/backend/tcop/postgres.c | 47 ++++++++++++++++++-----------
src/backend/utils/activity/pgstat.c | 2 +-
2 files changed, 31 insertions(+), 18 deletions(-)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8b6b5bbaaab..a3aa9063dc9 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3371,10 +3371,13 @@ ProcessInterrupts(void)
IdleSessionTimeoutPending = false;
}
- if (IdleStatsUpdateTimeoutPending)
+ /*
+ * If there are pending stats updates and we currently are truly idle
+ * (matching the conditions in PostgresMain(), report stats now.
+ */
+ if (IdleStatsUpdateTimeoutPending &&
+ DoingCommandRead && !IsTransactionOrTransactionBlock())
{
- /* timer should have been disarmed */
- Assert(!IsTransactionBlock());
IdleStatsUpdateTimeoutPending = false;
pgstat_report_stat(true);
}
@@ -4051,7 +4054,6 @@ PostgresMain(const char *dbname, const char *username)
volatile bool send_ready_for_query = true;
bool idle_in_transaction_timeout_enabled = false;
bool idle_session_timeout_enabled = false;
- bool idle_stats_update_timeout_enabled = false;
AssertArg(dbname != NULL);
AssertArg(username != NULL);
@@ -4428,13 +4430,30 @@ PostgresMain(const char *dbname, const char *username)
if (notifyInterruptPending)
ProcessNotifyInterrupt(false);
- /* Start the idle-stats-update timer */
+ /*
+ * Check if we need to report stats. If pgstat_report_stat()
+ * decides it's too soon to flush out pending stats / lock
+ * contention prevented reporting, it'll tell us when we
+ * should try to report stats again (so that stats updates
+ * aren't unduly delayed if the connection goes idle for a
+ * long time). We only enable the timeout if we don't already
+ * have a timeout in progress, because we don't disable the
+ * timeout below. enable_timeout_after() needs to determine
+ * the current timestamp, which can have a negative
+ * performance impact. That's OK because pgstat_report_stat()
+ * won't have us wake up sooner than a prior call.
+ */
stats_timeout = pgstat_report_stat(false);
if (stats_timeout > 0)
{
- idle_stats_update_timeout_enabled = true;
- enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
- stats_timeout);
+ if (!get_timeout_active(IDLE_STATS_UPDATE_TIMEOUT))
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
+ else
+ {
+ /* all stats flushed, no need for the timeout */
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
}
set_ps_display("idle");
@@ -4470,10 +4489,9 @@ PostgresMain(const char *dbname, const char *username)
firstchar = ReadCommand(&input_message);
/*
- * (4) turn off the idle-in-transaction, idle-session and
- * idle-stats-update timeouts if active. We do this before step (5)
- * so that any last-moment timeout is certain to be detected in step
- * (5).
+ * (4) turn off the idle-in-transaction and idle-session timeouts if
+ * active. We do this before step (5) so that any last-moment timeout
+ * is certain to be detected in step (5).
*
* At most one of these timeouts will be active, so there's no need to
* worry about combining the timeout.c calls into one.
@@ -4488,11 +4506,6 @@ PostgresMain(const char *dbname, const char *username)
disable_timeout(IDLE_SESSION_TIMEOUT, false);
idle_session_timeout_enabled = false;
}
- if (idle_stats_update_timeout_enabled)
- {
- disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
- idle_stats_update_timeout_enabled = false;
- }
/*
* (5) disable async signal conditions again.
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index ad5cf6fb915..826008be8b2 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -570,7 +570,7 @@ pgstat_report_stat(bool force)
bool nowait;
pgstat_assert_is_up();
- Assert(!IsTransactionBlock());
+ Assert(!IsTransactionOrTransactionBlock());
/* "absorb" the forced flush even if there's nothing to flush */
if (pgStatForceNextFlush)
--
2.35.1.677.gabf474a5dd
v2-0003-WIP-expression-eval-fix-EEOP_HASHED_SCALARARRAYOP.patch (text/x-diff, 5.5K) download
From c757eaaed0f3eb2e78a869f358111396716f81fa Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Thu, 16 Jun 2022 18:28:39 -0700
Subject: [PATCH v2 3/4] WIP: expression eval: fix EEOP_HASHED_SCALARARRAYOP
state width.
Author:
Reviewed-By:
Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de
Backpatch:
---
src/include/executor/execExpr.h | 5 -----
src/backend/executor/execExpr.c | 10 ----------
src/backend/executor/execExprInterp.c | 18 ++++++++++++++----
3 files changed, 14 insertions(+), 19 deletions(-)
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index e34db8c93cb..2fba1450c65 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -582,12 +582,7 @@ typedef struct ExprEvalStep
struct ScalarArrayOpExprHashTable *elements_tab;
FmgrInfo *finfo; /* function's lookup data */
FunctionCallInfo fcinfo_data; /* arguments etc */
- /* faster to access without additional indirection: */
- PGFunction fn_addr; /* actual call address */
FmgrInfo *hash_finfo; /* function's lookup data */
- FunctionCallInfo hash_fcinfo_data; /* arguments etc */
- /* faster to access without additional indirection: */
- PGFunction hash_fn_addr; /* actual call address */
} hashedscalararrayop;
/* for EEOP_XMLEXPR */
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 2831e7978b5..4128248be40 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -1204,7 +1204,6 @@ ExecInitExprRec(Expr *node, ExprState *state,
FunctionCallInfo fcinfo;
AclResult aclresult;
FmgrInfo *hash_finfo;
- FunctionCallInfo hash_fcinfo;
Oid cmpfuncid;
/*
@@ -1263,16 +1262,10 @@ ExecInitExprRec(Expr *node, ExprState *state,
if (OidIsValid(opexpr->hashfuncid))
{
hash_finfo = palloc0(sizeof(FmgrInfo));
- hash_fcinfo = palloc0(SizeForFunctionCallInfo(1));
fmgr_info(opexpr->hashfuncid, hash_finfo);
fmgr_info_set_expr((Node *) node, hash_finfo);
- InitFunctionCallInfoData(*hash_fcinfo, hash_finfo,
- 1, opexpr->inputcollid, NULL,
- NULL);
scratch.d.hashedscalararrayop.hash_finfo = hash_finfo;
- scratch.d.hashedscalararrayop.hash_fcinfo_data = hash_fcinfo;
- scratch.d.hashedscalararrayop.hash_fn_addr = hash_finfo->fn_addr;
/* Evaluate scalar directly into left function argument */
ExecInitExprRec(scalararg, state,
@@ -1292,11 +1285,8 @@ ExecInitExprRec(Expr *node, ExprState *state,
scratch.d.hashedscalararrayop.inclause = opexpr->useOr;
scratch.d.hashedscalararrayop.finfo = finfo;
scratch.d.hashedscalararrayop.fcinfo_data = fcinfo;
- scratch.d.hashedscalararrayop.fn_addr = finfo->fn_addr;
scratch.d.hashedscalararrayop.hash_finfo = hash_finfo;
- scratch.d.hashedscalararrayop.hash_fcinfo_data = hash_fcinfo;
- scratch.d.hashedscalararrayop.hash_fn_addr = hash_finfo->fn_addr;
ExprEvalPushStep(state, &scratch);
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index eaec697bb38..77e1cb1b7b4 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -217,6 +217,7 @@ typedef struct ScalarArrayOpExprHashTable
{
saophash_hash *hashtab; /* underlying hash table */
struct ExprEvalStep *op;
+ FunctionCallInfo hash_fcinfo_data; /* arguments etc */
} ScalarArrayOpExprHashTable;
/* Define parameters for ScalarArrayOpExpr hash table code generation. */
@@ -3474,13 +3475,13 @@ static uint32
saop_element_hash(struct saophash_hash *tb, Datum key)
{
ScalarArrayOpExprHashTable *elements_tab = (ScalarArrayOpExprHashTable *) tb->private_data;
- FunctionCallInfo fcinfo = elements_tab->op->d.hashedscalararrayop.hash_fcinfo_data;
+ FunctionCallInfo fcinfo = elements_tab->hash_fcinfo_data;
Datum hash;
fcinfo->args[0].value = key;
fcinfo->args[0].isnull = false;
- hash = elements_tab->op->d.hashedscalararrayop.hash_fn_addr(fcinfo);
+ hash = elements_tab->op->d.hashedscalararrayop.hash_finfo->fn_addr(fcinfo);
return DatumGetUInt32(hash);
}
@@ -3502,7 +3503,7 @@ saop_hash_element_match(struct saophash_hash *tb, Datum key1, Datum key2)
fcinfo->args[1].value = key2;
fcinfo->args[1].isnull = false;
- result = elements_tab->op->d.hashedscalararrayop.fn_addr(fcinfo);
+ result = elements_tab->op->d.hashedscalararrayop.finfo->fn_addr(fcinfo);
return DatumGetBool(result);
}
@@ -3575,6 +3576,15 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
op->d.hashedscalararrayop.elements_tab = elements_tab;
elements_tab->op = op;
+ elements_tab->hash_fcinfo_data = palloc0(SizeForFunctionCallInfo(1));
+ InitFunctionCallInfoData(*elements_tab->hash_fcinfo_data,
+ op->d.hashedscalararrayop.hash_finfo,
+ 1,
+ /* FIXME */
+ ((OpExpr*)op->d.hashedscalararrayop.hash_finfo->fn_expr)->inputcollid,
+ NULL,
+ NULL);
+
/*
* Create the hash table sizing it according to the number of elements
* in the array. This does assume that the array has no duplicates.
@@ -3669,7 +3679,7 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
fcinfo->args[1].value = (Datum) 0;
fcinfo->args[1].isnull = true;
- result = op->d.hashedscalararrayop.fn_addr(fcinfo);
+ result = op->d.hashedscalararrayop.finfo->fn_addr(fcinfo);
resultnull = fcinfo->isnull;
/*
--
2.35.1.677.gabf474a5dd
v2-0004-WIP-expression-eval-Fix-EEOP_JSON_CONSTRUCTOR-and.patch (text/x-diff, 20.8K) download
From 740b70d61f6f48929567dff84fc7ddcebef1551e Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Thu, 16 Jun 2022 18:33:42 -0700
Subject: [PATCH v2 4/4] WIP: expression eval: Fix EEOP_JSON_CONSTRUCTOR and
EEOP_JSONEXPR size.
Author:
Reviewed-By:
Discussion: https://postgr.es/m/
Backpatch:
---
src/include/executor/execExpr.h | 111 ++++++++++++++------------
src/backend/executor/execExpr.c | 93 +++++++++++----------
src/backend/executor/execExprInterp.c | 104 ++++++++++++------------
3 files changed, 168 insertions(+), 140 deletions(-)
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 2fba1450c65..95f05ddaa59 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -22,6 +22,8 @@ struct ExprEvalStep;
struct SubscriptingRefState;
struct ScalarArrayOpExprHashTable;
struct JsonbValue;
+struct JsonExprState;
+struct JsonConstructorExprState;
/* Bits in ExprState->flags (see also execnodes.h for public flag bits): */
/* expression's interpreter has been initialized */
@@ -671,16 +673,7 @@ typedef struct ExprEvalStep
/* for EEOP_JSON_CONSTRUCTOR */
struct
{
- JsonConstructorExpr *constructor;
- Datum *arg_values;
- bool *arg_nulls;
- Oid *arg_types;
- struct
- {
- int category;
- Oid outfuncid;
- } *arg_type_cache; /* cache for datum_to_json[b]() */
- int nargs;
+ struct JsonConstructorExprState *jcstate;
} json_constructor;
/* for EEOP_IS_JSON */
@@ -692,45 +685,7 @@ typedef struct ExprEvalStep
/* for EEOP_JSONEXPR */
struct
{
- JsonExpr *jsexpr; /* original expression node */
-
- struct
- {
- FmgrInfo func; /* typinput function for output type */
- Oid typioparam;
- } input; /* I/O info for output type */
-
- NullableDatum
- *formatted_expr, /* formatted context item value */
- *res_expr, /* result item */
- *coercion_expr, /* input for JSON item coercion */
- *pathspec; /* path specification value */
-
- ExprState *result_expr; /* coerced to output type */
- ExprState *default_on_empty; /* ON EMPTY DEFAULT expression */
- ExprState *default_on_error; /* ON ERROR DEFAULT expression */
- List *args; /* passing arguments */
-
- void *cache; /* cache for json_populate_type() */
-
- struct JsonCoercionsState
- {
- struct JsonCoercionState
- {
- JsonCoercion *coercion; /* coercion expression */
- ExprState *estate; /* coercion expression state */
- } null,
- string,
- numeric ,
- boolean,
- date,
- time,
- timetz,
- timestamp,
- timestamptz,
- composite;
- } coercions; /* states for coercion from SQL/JSON item
- * types directly to the output type */
+ struct JsonExprState *jsestate;
} jsonexpr;
} d;
@@ -777,6 +732,64 @@ typedef struct SubscriptExecSteps
ExecEvalSubroutine sbs_fetch_old; /* fetch old value for assignment */
} SubscriptExecSteps;
+/* EEOP_JSON_CONSTRUCTOR state, too big to inline */
+typedef struct JsonConstructorExprState
+{
+ JsonConstructorExpr *constructor;
+ Datum *arg_values;
+ bool *arg_nulls;
+ Oid *arg_types;
+ struct
+ {
+ int category;
+ Oid outfuncid;
+ } *arg_type_cache; /* cache for datum_to_json[b]() */
+ int nargs;
+} JsonConstructorExprState;
+
+/* EEOP_JSONEXPR state, too big to inline */
+typedef struct JsonExprState
+{
+ JsonExpr *jsexpr; /* original expression node */
+
+ struct
+ {
+ FmgrInfo func; /* typinput function for output type */
+ Oid typioparam;
+ } input; /* I/O info for output type */
+
+ NullableDatum
+ *formatted_expr, /* formatted context item value */
+ *res_expr, /* result item */
+ *coercion_expr, /* input for JSON item coercion */
+ *pathspec; /* path specification value */
+
+ ExprState *result_expr; /* coerced to output type */
+ ExprState *default_on_empty; /* ON EMPTY DEFAULT expression */
+ ExprState *default_on_error; /* ON ERROR DEFAULT expression */
+ List *args; /* passing arguments */
+
+ void *cache; /* cache for json_populate_type() */
+
+ struct JsonCoercionsState
+ {
+ struct JsonCoercionState
+ {
+ JsonCoercion *coercion; /* coercion expression */
+ ExprState *estate; /* coercion expression state */
+ } null,
+ string,
+ numeric ,
+ boolean,
+ date,
+ time,
+ timetz,
+ timestamp,
+ timestamptz,
+ composite;
+ } coercions; /* states for coercion from SQL/JSON item
+ * types directly to the output type */
+} JsonExprState;
/* functions in execExpr.c */
extern void ExprEvalPushStep(ExprState *es, const ExprEvalStep *s);
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 4128248be40..ca0b4cefb0f 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2460,32 +2460,38 @@ ExecInitExprRec(Expr *node, ExprState *state,
}
else
{
+ JsonConstructorExprState *jcstate;
+
+ jcstate = palloc0(sizeof(JsonConstructorExprState));
+
scratch.opcode = EEOP_JSON_CONSTRUCTOR;
- scratch.d.json_constructor.constructor = ctor;
- scratch.d.json_constructor.arg_values = palloc(sizeof(Datum) * nargs);
- scratch.d.json_constructor.arg_nulls = palloc(sizeof(bool) * nargs);
- scratch.d.json_constructor.arg_types = palloc(sizeof(Oid) * nargs);
- scratch.d.json_constructor.nargs = nargs;
+ scratch.d.json_constructor.jcstate = jcstate;
+
+ jcstate->constructor = ctor;
+ jcstate->arg_values = palloc(sizeof(Datum) * nargs);
+ jcstate->arg_nulls = palloc(sizeof(bool) * nargs);
+ jcstate->arg_types = palloc(sizeof(Oid) * nargs);
+ jcstate->nargs = nargs;
foreach(lc, args)
{
Expr *arg = (Expr *) lfirst(lc);
- scratch.d.json_constructor.arg_types[argno] = exprType((Node *) arg);
+ jcstate->arg_types[argno] = exprType((Node *) arg);
if (IsA(arg, Const))
{
/* Don't evaluate const arguments every round */
Const *con = (Const *) arg;
- scratch.d.json_constructor.arg_values[argno] = con->constvalue;
- scratch.d.json_constructor.arg_nulls[argno] = con->constisnull;
+ jcstate->arg_values[argno] = con->constvalue;
+ jcstate->arg_nulls[argno] = con->constisnull;
}
else
{
ExecInitExprRec(arg, state,
- &scratch.d.json_constructor.arg_values[argno],
- &scratch.d.json_constructor.arg_nulls[argno]);
+ &jcstate->arg_values[argno],
+ &jcstate->arg_nulls[argno]);
}
argno++;
}
@@ -2496,14 +2502,14 @@ ExecInitExprRec(Expr *node, ExprState *state,
bool is_jsonb =
ctor->returning->format->format_type == JS_FORMAT_JSONB;
- scratch.d.json_constructor.arg_type_cache =
- palloc(sizeof(*scratch.d.json_constructor.arg_type_cache) * nargs);
+ jcstate->arg_type_cache =
+ palloc(sizeof(*jcstate->arg_type_cache) * nargs);
for (int i = 0; i < nargs; i++)
{
int category;
Oid outfuncid;
- Oid typid = scratch.d.json_constructor.arg_types[i];
+ Oid typid = jcstate->arg_types[i];
if (is_jsonb)
{
@@ -2522,8 +2528,8 @@ ExecInitExprRec(Expr *node, ExprState *state,
category = (int) jscat;
}
- scratch.d.json_constructor.arg_type_cache[i].outfuncid = outfuncid;
- scratch.d.json_constructor.arg_type_cache[i].category = category;
+ jcstate->arg_type_cache[i].outfuncid = outfuncid;
+ jcstate->arg_type_cache[i].category = category;
}
}
@@ -2562,41 +2568,44 @@ ExecInitExprRec(Expr *node, ExprState *state,
case T_JsonExpr:
{
JsonExpr *jexpr = castNode(JsonExpr, node);
+ JsonExprState *jsestate = palloc0(sizeof(JsonExprState));
ListCell *argexprlc;
ListCell *argnamelc;
scratch.opcode = EEOP_JSONEXPR;
- scratch.d.jsonexpr.jsexpr = jexpr;
+ scratch.d.jsonexpr.jsestate = jsestate;
- scratch.d.jsonexpr.formatted_expr =
- palloc(sizeof(*scratch.d.jsonexpr.formatted_expr));
+ jsestate->jsexpr = jexpr;
+
+ jsestate->formatted_expr =
+ palloc(sizeof(*jsestate->formatted_expr));
ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
- &scratch.d.jsonexpr.formatted_expr->value,
- &scratch.d.jsonexpr.formatted_expr->isnull);
+ &jsestate->formatted_expr->value,
+ &jsestate->formatted_expr->isnull);
- scratch.d.jsonexpr.pathspec =
- palloc(sizeof(*scratch.d.jsonexpr.pathspec));
+ jsestate->pathspec =
+ palloc(sizeof(*jsestate->pathspec));
ExecInitExprRec((Expr *) jexpr->path_spec, state,
- &scratch.d.jsonexpr.pathspec->value,
- &scratch.d.jsonexpr.pathspec->isnull);
+ &jsestate->pathspec->value,
+ &jsestate->pathspec->isnull);
- scratch.d.jsonexpr.res_expr =
- palloc(sizeof(*scratch.d.jsonexpr.res_expr));
+ jsestate->res_expr =
+ palloc(sizeof(*jsestate->res_expr));
- scratch.d.jsonexpr.result_expr = jexpr->result_coercion
+ jsestate->result_expr = jexpr->result_coercion
? ExecInitExprWithCaseValue((Expr *) jexpr->result_coercion->expr,
state->parent,
- &scratch.d.jsonexpr.res_expr->value,
- &scratch.d.jsonexpr.res_expr->isnull)
+ &jsestate->res_expr->value,
+ &jsestate->res_expr->isnull)
: NULL;
- scratch.d.jsonexpr.default_on_empty = !jexpr->on_empty ? NULL :
+ jsestate->default_on_empty = !jexpr->on_empty ? NULL :
ExecInitExpr((Expr *) jexpr->on_empty->default_expr,
state->parent);
- scratch.d.jsonexpr.default_on_error =
+ jsestate->default_on_error =
ExecInitExpr((Expr *) jexpr->on_error->default_expr,
state->parent);
@@ -2607,11 +2616,11 @@ ExecInitExprRec(Expr *node, ExprState *state,
/* lookup the result type's input function */
getTypeInputInfo(jexpr->returning->typid, &typinput,
- &scratch.d.jsonexpr.input.typioparam);
- fmgr_info(typinput, &scratch.d.jsonexpr.input.func);
+ &jsestate->input.typioparam);
+ fmgr_info(typinput, &jsestate->input.func);
}
- scratch.d.jsonexpr.args = NIL;
+ jsestate->args = NIL;
forboth(argexprlc, jexpr->passing_values,
argnamelc, jexpr->passing_names)
@@ -2630,11 +2639,11 @@ ExecInitExprRec(Expr *node, ExprState *state,
var->value = (Datum) 0;
var->isnull = true;
- scratch.d.jsonexpr.args =
- lappend(scratch.d.jsonexpr.args, var);
+ jsestate->args =
+ lappend(jsestate->args, var);
}
- scratch.d.jsonexpr.cache = NULL;
+ jsestate->cache = NULL;
if (jexpr->coercions)
{
@@ -2643,13 +2652,13 @@ ExecInitExprRec(Expr *node, ExprState *state,
Datum *caseval;
bool *casenull;
- scratch.d.jsonexpr.coercion_expr =
- palloc(sizeof(*scratch.d.jsonexpr.coercion_expr));
+ jsestate->coercion_expr =
+ palloc(sizeof(*jsestate->coercion_expr));
- caseval = &scratch.d.jsonexpr.coercion_expr->value;
- casenull = &scratch.d.jsonexpr.coercion_expr->isnull;
+ caseval = &jsestate->coercion_expr->value;
+ casenull = &jsestate->coercion_expr->isnull;
- for (cstate = &scratch.d.jsonexpr.coercions.null,
+ for (cstate = &jsestate->coercions.null,
coercion = &jexpr->coercions->null;
coercion <= &jexpr->coercions->composite;
coercion++, cstate++)
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 77e1cb1b7b4..9913907cec8 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -4520,39 +4520,40 @@ ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
ExprContext *econtext)
{
Datum res;
- JsonConstructorExpr *ctor = op->d.json_constructor.constructor;
+ JsonConstructorExprState *jcstate = op->d.json_constructor.jcstate;
+ JsonConstructorExpr *ctor = jcstate->constructor;
bool is_jsonb = ctor->returning->format->format_type == JS_FORMAT_JSONB;
bool isnull = false;
if (ctor->type == JSCTOR_JSON_ARRAY)
res = (is_jsonb ?
jsonb_build_array_worker :
- json_build_array_worker) (op->d.json_constructor.nargs,
- op->d.json_constructor.arg_values,
- op->d.json_constructor.arg_nulls,
- op->d.json_constructor.arg_types,
- op->d.json_constructor.constructor->absent_on_null);
+ json_build_array_worker) (jcstate->nargs,
+ jcstate->arg_values,
+ jcstate->arg_nulls,
+ jcstate->arg_types,
+ ctor->absent_on_null);
else if (ctor->type == JSCTOR_JSON_OBJECT)
res = (is_jsonb ?
jsonb_build_object_worker :
- json_build_object_worker) (op->d.json_constructor.nargs,
- op->d.json_constructor.arg_values,
- op->d.json_constructor.arg_nulls,
- op->d.json_constructor.arg_types,
- op->d.json_constructor.constructor->absent_on_null,
- op->d.json_constructor.constructor->unique);
+ json_build_object_worker) (jcstate->nargs,
+ jcstate->arg_values,
+ jcstate->arg_nulls,
+ jcstate->arg_types,
+ ctor->absent_on_null,
+ ctor->unique);
else if (ctor->type == JSCTOR_JSON_SCALAR)
{
- if (op->d.json_constructor.arg_nulls[0])
+ if (jcstate->arg_nulls[0])
{
res = (Datum) 0;
isnull = true;
}
else
{
- Datum value = op->d.json_constructor.arg_values[0];
- int category = op->d.json_constructor.arg_type_cache[0].category;
- Oid outfuncid = op->d.json_constructor.arg_type_cache[0].outfuncid;
+ Datum value = jcstate->arg_values[0];
+ int category = jcstate->arg_type_cache[0].category;
+ Oid outfuncid = jcstate->arg_type_cache[0].outfuncid;
if (is_jsonb)
res = to_jsonb_worker(value, category, outfuncid);
@@ -4562,14 +4563,14 @@ ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
}
else if (ctor->type == JSCTOR_JSON_PARSE)
{
- if (op->d.json_constructor.arg_nulls[0])
+ if (jcstate->arg_nulls[0])
{
res = (Datum) 0;
isnull = true;
}
else
{
- Datum value = op->d.json_constructor.arg_values[0];
+ Datum value = jcstate->arg_values[0];
text *js = DatumGetTextP(value);
if (is_jsonb)
@@ -4637,14 +4638,17 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
Datum res, bool *isNull, void *p, bool *error)
{
ExprState *estate = p;
+ JsonExprState *jsestate;
if (estate) /* coerce using specified expression */
return ExecEvalExpr(estate, econtext, isNull);
- if (op->d.jsonexpr.jsexpr->op != JSON_EXISTS_OP)
+ jsestate = op->d.jsonexpr.jsestate;
+
+ if (jsestate->jsexpr->op != JSON_EXISTS_OP)
{
- JsonCoercion *coercion = op->d.jsonexpr.jsexpr->result_coercion;
- JsonExpr *jexpr = op->d.jsonexpr.jsexpr;
+ JsonCoercion *coercion = jsestate->jsexpr->result_coercion;
+ JsonExpr *jexpr = jsestate->jsexpr;
Jsonb *jb = *isNull ? NULL : DatumGetJsonbP(res);
if ((coercion && coercion->via_io) ||
@@ -4654,25 +4658,25 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
/* strip quotes and call typinput function */
char *str = *isNull ? NULL : JsonbUnquote(jb);
- return InputFunctionCall(&op->d.jsonexpr.input.func, str,
- op->d.jsonexpr.input.typioparam,
+ return InputFunctionCall(&jsestate->input.func, str,
+ jsestate->input.typioparam,
jexpr->returning->typmod);
}
else if (coercion && coercion->via_populate)
return json_populate_type(res, JSONBOID,
jexpr->returning->typid,
jexpr->returning->typmod,
- &op->d.jsonexpr.cache,
+ &jsestate->cache,
econtext->ecxt_per_query_memory,
isNull);
}
- if (op->d.jsonexpr.result_expr)
+ if (jsestate->result_expr)
{
- op->d.jsonexpr.res_expr->value = res;
- op->d.jsonexpr.res_expr->isnull = *isNull;
+ jsestate->res_expr->value = res;
+ jsestate->res_expr->isnull = *isNull;
- res = ExecEvalExpr(op->d.jsonexpr.result_expr, econtext, isNull);
+ res = ExecEvalExpr(jsestate->result_expr, econtext, isNull);
}
return res;
@@ -4905,7 +4909,8 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
{
ExecEvalJsonExprContext *cxt = pcxt;
JsonPath *path = cxt->path;
- JsonExpr *jexpr = op->d.jsonexpr.jsexpr;
+ JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExpr *jexpr = jsestate->jsexpr;
ExprState *estate = NULL;
bool empty = false;
Datum res = (Datum) 0;
@@ -4914,7 +4919,7 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
{
case JSON_QUERY_OP:
res = JsonPathQuery(item, path, jexpr->wrapper, &empty, error,
- op->d.jsonexpr.args);
+ jsestate->args);
if (error && *error)
{
*resnull = true;
@@ -4927,7 +4932,7 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
{
struct JsonCoercionState *jcstate;
JsonbValue *jbv = JsonPathValue(item, path, &empty, error,
- op->d.jsonexpr.args);
+ jsestate->args);
if (error && *error)
return (Datum) 0;
@@ -4950,8 +4955,8 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
/* Use coercion from SQL/JSON item type to the output type */
res = ExecPrepareJsonItemCoercion(jbv,
- op->d.jsonexpr.jsexpr->returning,
- &op->d.jsonexpr.coercions,
+ jsestate->jsexpr->returning,
+ &jsestate->coercions,
&jcstate);
if (jcstate->coercion &&
@@ -4983,27 +4988,27 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
/* coerce using specific expression */
estate = jcstate->estate;
- op->d.jsonexpr.coercion_expr->value = res;
- op->d.jsonexpr.coercion_expr->isnull = *resnull;
+ jsestate->coercion_expr->value = res;
+ jsestate->coercion_expr->isnull = *resnull;
break;
}
case JSON_EXISTS_OP:
{
bool exists = JsonPathExists(item, path,
- op->d.jsonexpr.args,
+ jsestate->args,
error);
*resnull = error && *error;
res = BoolGetDatum(exists);
- if (!op->d.jsonexpr.result_expr)
+ if (!jsestate->result_expr)
return res;
/* coerce using result expression */
- estate = op->d.jsonexpr.result_expr;
- op->d.jsonexpr.res_expr->value = res;
- op->d.jsonexpr.res_expr->isnull = *resnull;
+ estate = jsestate->result_expr;
+ jsestate->res_expr->value = res;
+ jsestate->res_expr->isnull = *resnull;
break;
}
@@ -5039,11 +5044,11 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
* Execute DEFAULT expression as a coercion expression, because
* its result is already coerced to the target type.
*/
- estate = op->d.jsonexpr.default_on_empty;
+ estate = jsestate->default_on_empty;
else
/* Execute ON EMPTY behavior */
res = ExecEvalJsonBehavior(econtext, jexpr->on_empty,
- op->d.jsonexpr.default_on_empty,
+ jsestate->default_on_empty,
resnull);
}
@@ -5076,7 +5081,8 @@ void
ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
{
ExecEvalJsonExprContext cxt;
- JsonExpr *jexpr = op->d.jsonexpr.jsexpr;
+ JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExpr *jexpr = jsestate->jsexpr;
Datum item;
Datum res = (Datum) 0;
JsonPath *path;
@@ -5088,7 +5094,7 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
*op->resnull = true; /* until we get a result */
*op->resvalue = (Datum) 0;
- if (op->d.jsonexpr.formatted_expr->isnull || op->d.jsonexpr.pathspec->isnull)
+ if (jsestate->formatted_expr->isnull || jsestate->pathspec->isnull)
{
/* execute domain checks for NULLs */
(void) ExecEvalJsonExprCoercion(op, econtext, res, op->resnull,
@@ -5098,11 +5104,11 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
return;
}
- item = op->d.jsonexpr.formatted_expr->value;
- path = DatumGetJsonPathP(op->d.jsonexpr.pathspec->value);
+ item = jsestate->formatted_expr->value;
+ path = DatumGetJsonPathP(jsestate->pathspec->value);
/* reset JSON path variable contexts */
- foreach(lc, op->d.jsonexpr.args)
+ foreach(lc, jsestate->args)
{
JsonPathVariableEvalContext *var = lfirst(lc);
@@ -5110,7 +5116,7 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
var->evaluated = false;
}
- needSubtrans = ExecEvalJsonNeedsSubTransaction(jexpr, &op->d.jsonexpr.coercions);
+ needSubtrans = ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
cxt.path = path;
cxt.error = throwErrors ? NULL : &error;
@@ -5125,7 +5131,7 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
{
/* Execute ON ERROR behavior */
res = ExecEvalJsonBehavior(econtext, jexpr->on_error,
- op->d.jsonexpr.default_on_error,
+ jsestate->default_on_error,
op->resnull);
/* result is already coerced in DEFAULT behavior case */
--
2.35.1.677.gabf474a5dd
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-21 21:11 ` Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 75+ messages in thread
From: Andrew Dunstan @ 2022-06-21 21:11 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-06-17 Fr 16:06, Andres Freund wrote:
>
>
> I also attached my heavily-WIP patches for the ExprEvalStep issues,
Many thanks
> I
> accidentally had only included a small part of the contents of the json fix.
>
Yeah, that confused me mightily last week :-)
I and a couple of colleagues have looked it over. As far as it goes the
json fix looks kosher to me. I'll play with it some more.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-06-21 21:25 ` Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
0 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-06-21 21:25 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-06-21 17:11:33 -0400, Andrew Dunstan wrote:
> I and a couple of colleagues have looked it over. As far as it goes the
> json fix looks kosher to me. I'll play with it some more.
Cool.
Any chance you could look at fixing the "structure" of the generated
expression "program". The recursive ExecEvalExpr() calls are really not ok...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-21 21:41 ` Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 75+ messages in thread
From: Andrew Dunstan @ 2022-06-21 21:41 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-06-21 Tu 17:25, Andres Freund wrote:
> Hi,
>
> On 2022-06-21 17:11:33 -0400, Andrew Dunstan wrote:
>> I and a couple of colleagues have looked it over. As far as it goes the
>> json fix looks kosher to me. I'll play with it some more.
> Cool.
>
> Any chance you could look at fixing the "structure" of the generated
> expression "program". The recursive ExecEvalExpr() calls are really not ok...
>
Yes, but I don't guarantee to have a fix in time for Beta2.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-06-23 07:38 ` Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 75+ messages in thread
From: Michael Paquier @ 2022-06-23 07:38 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On Tue, Jun 21, 2022 at 05:41:07PM -0400, Andrew Dunstan wrote:
> On 2022-06-21 Tu 17:25, Andres Freund wrote:
>> On 2022-06-21 17:11:33 -0400, Andrew Dunstan wrote:
>>> I and a couple of colleagues have looked it over. As far as it goes the
>>> json fix looks kosher to me. I'll play with it some more.
>>
>> Cool.
>>
>> Any chance you could look at fixing the "structure" of the generated
>> expression "program". The recursive ExecEvalExpr() calls are really not ok...
By how much does the size of ExprEvalStep go down once you don't
inline the JSON structures as of 0004 in [1]? And what of 0003? The
JSON portions seem like the largest portion of the cake, though both
are must-fixes.
> Yes, but I don't guarantee to have a fix in time for Beta2.
IMHO, it would be nice to get something done for beta2. Now the
thread is rather fresh and I guess that more performance study is
required even for 0004, so.. Waiting for beta3 would a better move at
this stage. Is somebody confident enough in the patches proposed?
0004 looks rather sane, seen from here, at least.
[1]: https://www.postgresql.org/message-id/20220617200605.3moq7dtxua5cxemv@alap3.anarazel.de
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
@ 2022-06-24 01:51 ` Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 19:08 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 75+ messages in thread
From: Andres Freund @ 2022-06-24 01:51 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-06-23 16:38:12 +0900, Michael Paquier wrote:
> On Tue, Jun 21, 2022 at 05:41:07PM -0400, Andrew Dunstan wrote:
> > On 2022-06-21 Tu 17:25, Andres Freund wrote:
> >> On 2022-06-21 17:11:33 -0400, Andrew Dunstan wrote:
> >>> I and a couple of colleagues have looked it over. As far as it goes the
> >>> json fix looks kosher to me. I'll play with it some more.
> >>
> >> Cool.
> >>
> >> Any chance you could look at fixing the "structure" of the generated
> >> expression "program". The recursive ExecEvalExpr() calls are really not ok...
>
> By how much does the size of ExprEvalStep go down once you don't
> inline the JSON structures as of 0004 in [1]? And what of 0003?
0004 gets us back to 64 bytes, if 0003 is applied first. 0003 alone doesn't
yield a size reduction, because obviously 0004 is the bigger problem. Applying
just 0004 you end up with 88 bytes.
> The JSON portions seem like the largest portion of the cake, though both are
> must-fixes.
Yep.
> > Yes, but I don't guarantee to have a fix in time for Beta2.
>
> IMHO, it would be nice to get something done for beta2. Now the
> thread is rather fresh and I guess that more performance study is
> required even for 0004, so..
I don't think there's a whole lot of performance study needed for 0004 - the
current state is obviously wrong.
I think Andrew's beta 2 comment was more about my other architectural
complains around the json expression eval stuff.
> Waiting for beta3 would a better move at this stage. Is somebody confident
> enough in the patches proposed?
0001 is the one that needs to most careful analysis, I think. 0002 I'd be fine
with pushing after reviewing it again. For 0003 David's approach might be
better or worse, it doesn't matter much I think. 0004 is ok I think, perhaps
with the exception of quibbling over some naming decisions?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-24 14:29 ` Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 75+ messages in thread
From: Andrew Dunstan @ 2022-06-24 14:29 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Michael Paquier <michael@paquier.xyz>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-06-23 Th 21:51, Andres Freund wrote:
> Hi,
>
> On 2022-06-23 16:38:12 +0900, Michael Paquier wrote:
>> On Tue, Jun 21, 2022 at 05:41:07PM -0400, Andrew Dunstan wrote:
>>> On 2022-06-21 Tu 17:25, Andres Freund wrote:
>>>> On 2022-06-21 17:11:33 -0400, Andrew Dunstan wrote:
>>>>> I and a couple of colleagues have looked it over. As far as it goes the
>>>>> json fix looks kosher to me. I'll play with it some more.
>>>> Cool.
>>>>
>>>> Any chance you could look at fixing the "structure" of the generated
>>>> expression "program". The recursive ExecEvalExpr() calls are really not ok...
>> By how much does the size of ExprEvalStep go down once you don't
>> inline the JSON structures as of 0004 in [1]? And what of 0003?
> 0004 gets us back to 64 bytes, if 0003 is applied first. 0003 alone doesn't
> yield a size reduction, because obviously 0004 is the bigger problem. Applying
> just 0004 you end up with 88 bytes.
>
>
>> The JSON portions seem like the largest portion of the cake, though both are
>> must-fixes.
> Yep.
>
>
>>> Yes, but I don't guarantee to have a fix in time for Beta2.
>> IMHO, it would be nice to get something done for beta2. Now the
>> thread is rather fresh and I guess that more performance study is
>> required even for 0004, so..
> I don't think there's a whole lot of performance study needed for 0004 - the
> current state is obviously wrong.
>
> I think Andrew's beta 2 comment was more about my other architectural
> complains around the json expression eval stuff.
Right. That's being worked on but it's not going to be a mechanical fix.
>
>
>> Waiting for beta3 would a better move at this stage. Is somebody confident
>> enough in the patches proposed?
> 0001 is the one that needs to most careful analysis, I think. 0002 I'd be fine
> with pushing after reviewing it again. For 0003 David's approach might be
> better or worse, it doesn't matter much I think. 0004 is ok I think, perhaps
> with the exception of quibbling over some naming decisions?
>
>
The attached very small patch applies on top of your 0002 and deals with
the FmgrInfo complaint.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Attachments:
0002-in-JsonExprState-just-store-a-pointer-to-the-input-F.patch (text/x-patch, 2.2K) download
From ed2118c63e298e5e5872db5003fa342c33aa7505 Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Wed, 22 Jun 2022 12:00:47 -0400
Subject: [PATCH 2/2] in JsonExprState just store a pointer to the input
FmgrInfo
---
src/backend/executor/execExpr.c | 5 ++++-
src/backend/executor/execExprInterp.c | 2 +-
src/include/executor/execExpr.h | 2 +-
3 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 1f65daf203..d8608a3c58 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2623,11 +2623,14 @@ ExecInitExprRec(Expr *node, ExprState *state,
(jexpr->result_coercion && jexpr->result_coercion->via_io))
{
Oid typinput;
+ FmgrInfo *finfo;
/* lookup the result type's input function */
getTypeInputInfo(jexpr->returning->typid, &typinput,
&jsestate->input.typioparam);
- fmgr_info(typinput, &jsestate->input.func);
+ finfo = palloc0(sizeof(FmgrInfo));
+ fmgr_info(typinput, finfo);
+ jsestate->input.finfo = finfo;
}
jsestate->args = NIL;
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 80c5e04dd5..cbdcbbce53 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -4648,7 +4648,7 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
/* strip quotes and call typinput function */
char *str = *isNull ? NULL : JsonbUnquote(jb);
- return InputFunctionCall(&jsestate->input.func, str,
+ return InputFunctionCall(jsestate->input.finfo, str,
jsestate->input.typioparam,
jexpr->returning->typmod);
}
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 15532813ae..c0ba8998f2 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -759,7 +759,7 @@ typedef struct JsonExprState
struct
{
- FmgrInfo func; /* typinput function for output type */
+ FmgrInfo *finfo; /* typinput function for output type */
Oid typioparam;
} input; /* I/O info for output type */
--
2.34.1
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-07-05 18:36 ` Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
0 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-07-05 18:36 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Michael Paquier <michael@paquier.xyz>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-06-24 10:29:06 -0400, Andrew Dunstan wrote:
> On 2022-06-23 Th 21:51, Andres Freund wrote:
> > On 2022-06-23 16:38:12 +0900, Michael Paquier wrote:
> >> On Tue, Jun 21, 2022 at 05:41:07PM -0400, Andrew Dunstan wrote:
> >>> Yes, but I don't guarantee to have a fix in time for Beta2.
> >> IMHO, it would be nice to get something done for beta2. Now the
> >> thread is rather fresh and I guess that more performance study is
> >> required even for 0004, so..
> > I don't think there's a whole lot of performance study needed for 0004 - the
> > current state is obviously wrong.
> >
> > I think Andrew's beta 2 comment was more about my other architectural
> > complains around the json expression eval stuff.
>
>
> Right. That's being worked on but it's not going to be a mechanical fix.
Any updates here?
I'd mentioned the significant space use due to all JsonCoercionsState for all
the types. Another related aspect is that this code is just weird - the same
struct name (JsonCoercionsState), nested in each other?
struct JsonCoercionsState
{
struct JsonCoercionState
{
JsonCoercion *coercion; /* coercion expression */
ExprState *estate; /* coercion expression state */
} null,
string,
numeric ,
boolean,
date,
time,
timetz,
timestamp,
timestamptz,
composite;
} coercions; /* states for coercion from SQL/JSON item
* types directly to the output type */
Also note the weird numeric indentation that pgindent does...
> The attached very small patch applies on top of your 0002 and deals with
> the FmgrInfo complaint.
Now that the FmgrInfo is part of a separately allocated struct, that doesn't
seem necessary anymore.
- Andres
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-07-05 19:04 ` Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
0 siblings, 1 reply; 75+ messages in thread
From: Andrew Dunstan @ 2022-07-05 19:04 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Michael Paquier <michael@paquier.xyz>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-07-05 Tu 14:36, Andres Freund wrote:
> Hi,
>
> On 2022-06-24 10:29:06 -0400, Andrew Dunstan wrote:
>> On 2022-06-23 Th 21:51, Andres Freund wrote:
>>> On 2022-06-23 16:38:12 +0900, Michael Paquier wrote:
>>>> On Tue, Jun 21, 2022 at 05:41:07PM -0400, Andrew Dunstan wrote:
>>>>> Yes, but I don't guarantee to have a fix in time for Beta2.
>>>> IMHO, it would be nice to get something done for beta2. Now the
>>>> thread is rather fresh and I guess that more performance study is
>>>> required even for 0004, so..
>>> I don't think there's a whole lot of performance study needed for 0004 - the
>>> current state is obviously wrong.
>>>
>>> I think Andrew's beta 2 comment was more about my other architectural
>>> complains around the json expression eval stuff.
>>
>> Right. That's being worked on but it's not going to be a mechanical fix.
> Any updates here?
Not yet. A colleague and I are working on it. I'll post a status this
week if we can't post a fix.
>
> I'd mentioned the significant space use due to all JsonCoercionsState for all
> the types. Another related aspect is that this code is just weird - the same
> struct name (JsonCoercionsState), nested in each other?
>
> struct JsonCoercionsState
> {
> struct JsonCoercionState
> {
> JsonCoercion *coercion; /* coercion expression */
> ExprState *estate; /* coercion expression state */
> } null,
> string,
> numeric ,
> boolean,
> date,
> time,
> timetz,
> timestamp,
> timestamptz,
> composite;
> } coercions; /* states for coercion from SQL/JSON item
> * types directly to the output type */
>
> Also note the weird numeric indentation that pgindent does...
Yeah, we'll try to fix that.
>
>
>> The attached very small patch applies on top of your 0002 and deals with
>> the FmgrInfo complaint.
> Now that the FmgrInfo is part of a separately allocated struct, that doesn't
> seem necessary anymore.
Right, but you complained that we should do it the same way as it's done
elsewhere, so I thought I'd do that anyway.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-07-08 21:05 ` Andrew Dunstan <andrew@dunslane.net>
2022-07-11 00:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 75+ messages in thread
From: Andrew Dunstan @ 2022-07-08 21:05 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Michael Paquier <michael@paquier.xyz>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-07-05 Tu 15:04, Andrew Dunstan wrote:
> On 2022-07-05 Tu 14:36, Andres Freund wrote:
>>
>>>> I think Andrew's beta 2 comment was more about my other architectural
>>>> complains around the json expression eval stuff.
>>> Right. That's being worked on but it's not going to be a mechanical fix.
>> Any updates here?
>
> Not yet. A colleague and I are working on it. I'll post a status this
> week if we can't post a fix.
We're still working on it. We've made substantial progress but there are
some tests failing that we need to fix.
>> I'd mentioned the significant space use due to all JsonCoercionsState for all
>> the types. Another related aspect is that this code is just weird - the same
>> struct name (JsonCoercionsState), nested in each other?
>>
>> struct JsonCoercionsState
>> {
>> struct JsonCoercionState
>> {
>> JsonCoercion *coercion; /* coercion expression */
>> ExprState *estate; /* coercion expression state */
>> } null,
>> string,
>> numeric ,
>> boolean,
>> date,
>> time,
>> timetz,
>> timestamp,
>> timestamptz,
>> composite;
>> } coercions; /* states for coercion from SQL/JSON item
>> * types directly to the output type */
>>
>> Also note the weird numeric indentation that pgindent does...
>
> Yeah, we'll try to fix that.
Actually, it's not the same name: JsonCoercionsState vs
JsonCoercionState. But I agree that it's a subtle enough difference that
we should use something more obvious. Maybe JsonCoercionStates instead
of JsonCoercionsState? The plural at the end would be harder to miss.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-07-11 00:29 ` Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 75+ messages in thread
From: Andres Freund @ 2022-07-11 00:29 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Michael Paquier <michael@paquier.xyz>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-07-08 17:05:49 -0400, Andrew Dunstan wrote:
> Actually, it's not the same name: JsonCoercionsState vs
> JsonCoercionState. But I agree that it's a subtle enough difference that
> we should use something more obvious. Maybe JsonCoercionStates instead
> of JsonCoercionsState? The plural at the end would be harder to miss.
Given that it's a one-off use struct, why name it? Then we don't have to
figure out a name we never use.
I also still would like to understand why we need pre-allocated space for all
these types. How could multiple datums be coerced in an interleaved manner?
And if that's possible, why can't multiple datums of the same type be coerced
at the same time?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-07-15 21:07 ` Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
1 sibling, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-07-15 21:07 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-07-08 17:05:49 -0400, Andrew Dunstan wrote:
> On 2022-07-05 Tu 15:04, Andrew Dunstan wrote:
> > On 2022-07-05 Tu 14:36, Andres Freund wrote:
> >>
> >>>> I think Andrew's beta 2 comment was more about my other architectural
> >>>> complains around the json expression eval stuff.
> >>> Right. That's being worked on but it's not going to be a mechanical fix.
> >> Any updates here?
> >
> > Not yet. A colleague and I are working on it. I'll post a status this
> > week if we can't post a fix.
> We're still working on it. We've made substantial progress but there are
> some tests failing that we need to fix.
I think we need to resolve this soon - or consider the alternatives. A lot of
the new json stuff doesn't seem fully baked, so I'm starting to wonder if we
have to consider pushing it a release further down.
Perhaps you could post your current state? I might be able to help resolving
some of the problems.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-07-18 19:09 ` Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Andrew Dunstan @ 2022-07-18 19:09 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-07-15 Fr 17:07, Andres Freund wrote:
> Hi,
>
> On 2022-07-08 17:05:49 -0400, Andrew Dunstan wrote:
>> On 2022-07-05 Tu 15:04, Andrew Dunstan wrote:
>>> On 2022-07-05 Tu 14:36, Andres Freund wrote:
>>>>>> I think Andrew's beta 2 comment was more about my other architectural
>>>>>> complains around the json expression eval stuff.
>>>>> Right. That's being worked on but it's not going to be a mechanical fix.
>>>> Any updates here?
>>> Not yet. A colleague and I are working on it. I'll post a status this
>>> week if we can't post a fix.
>> We're still working on it. We've made substantial progress but there are
>> some tests failing that we need to fix.
> I think we need to resolve this soon - or consider the alternatives. A lot of
> the new json stuff doesn't seem fully baked, so I'm starting to wonder if we
> have to consider pushing it a release further down.
>
> Perhaps you could post your current state? I might be able to help resolving
> some of the problems.
Ok. Here is the state of things. This has proved to be rather more
intractable than I expected. Almost all the legwork here has been done
by Amit Langote, for which he deserves both my thanks and considerable
credit, but I take responsibility for it.
I just discovered today that this scheme is failing under
"force_parallel_mode = regress". I have as yet no idea if that can be
fixed simply or not. Apart from that I think the main outstanding issue
is to fill in the gaps in llvm_compile_expr().
If you have help you can offer that would be very welcome.
I'd still very much like to get this done, but if the decision is we've
run out of time I'll be sad but understand.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Attachments: download all (3 files, .tar.gz)
0001-in-JsonExprState-just-store-a-pointer-to-the-input-F.patch (text/x-patch, 2.2K) download
From 7c0d831f3baf6fb6ec27d1e033209535945e4858 Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Mon, 18 Jul 2022 11:03:13 -0400
Subject: [PATCH 1/3] in JsonExprState just store a pointer to the input
FmgrInfo
---
src/backend/executor/execExpr.c | 5 ++++-
src/backend/executor/execExprInterp.c | 2 +-
src/include/executor/execExpr.h | 2 +-
3 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index c8d7145fe3..a55e5000e2 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2606,11 +2606,14 @@ ExecInitExprRec(Expr *node, ExprState *state,
(jexpr->result_coercion && jexpr->result_coercion->via_io))
{
Oid typinput;
+ FmgrInfo *finfo;
/* lookup the result type's input function */
getTypeInputInfo(jexpr->returning->typid, &typinput,
&jsestate->input.typioparam);
- fmgr_info(typinput, &jsestate->input.func);
+ finfo = palloc0(sizeof(FmgrInfo));
+ fmgr_info(typinput, finfo);
+ jsestate->input.finfo = finfo;
}
jsestate->args = NIL;
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 723770fda0..0512a81c7c 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -4664,7 +4664,7 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
/* strip quotes and call typinput function */
char *str = *isNull ? NULL : JsonbUnquote(jb);
- return InputFunctionCall(&jsestate->input.func, str,
+ return InputFunctionCall(jsestate->input.finfo, str,
jsestate->input.typioparam,
jexpr->returning->typmod);
}
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 1e3f1bbee8..e55a572854 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -754,7 +754,7 @@ typedef struct JsonExprState
struct
{
- FmgrInfo func; /* typinput function for output type */
+ FmgrInfo *finfo; /* typinput function for output type */
Oid typioparam;
} input; /* I/O info for output type */
--
2.34.1
0002-Evaluate-various-JsonExpr-sub-expressions-using-pare.patch (text/x-patch, 35.5K) download
From ee39dadaa18f45c59224d4da908b36db7a2a8b0f Mon Sep 17 00:00:00 2001
From: amitlan <amitlangote09@gmail.com>
Date: Mon, 18 Jul 2022 11:04:17 -0400
Subject: [PATCH 2/3] Evaluate various JsonExpr sub-expressions using parent
ExprState
These include PASSING args, ON ERROR, ON EMPTY default expressions,
and result_coercion expression.
To do so, this moves a bunch of code from ExecEvalJson(),
ExecEvalJsonExpr(), and ExecEvalJsonExprCoercion(), all of which
would previously run under the single step EEOP_JSONEXPR steps into
a number of new (sub-) ExprEvalSteps that are now added for a given
JsonExpr.
TODO: update llvm_compile_expr() to handle newly added
EEOP_JSONEXPR_* steps.
---
src/backend/executor/execExpr.c | 288 +++++++++++++++-----
src/backend/executor/execExprInterp.c | 366 ++++++++++++++++++--------
src/backend/jit/llvm/llvmjit_expr.c | 35 ++-
src/include/executor/execExpr.h | 136 ++++++++--
4 files changed, 639 insertions(+), 186 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index a55e5000e2..4dbb24aaee 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2562,62 +2562,47 @@ ExecInitExprRec(Expr *node, ExprState *state,
{
JsonExpr *jexpr = castNode(JsonExpr, node);
JsonExprState *jsestate = palloc0(sizeof(JsonExprState));
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
ListCell *argexprlc;
ListCell *argnamelc;
-
- scratch.opcode = EEOP_JSONEXPR;
- scratch.d.jsonexpr.jsestate = jsestate;
+ int skip_step_off;
+ int onempty_step_off;
+ int onerror_step_off;
+ int coercion_step_off;
+ int coercion_error_step_off;
+ int result_step_off;
+ ExprEvalStep *as;
+ bool throwErrors =
+ (jexpr->on_error->btype == JSON_BEHAVIOR_ERROR);
jsestate->jsexpr = jexpr;
- jsestate->formatted_expr =
- palloc(sizeof(*jsestate->formatted_expr));
-
+ /*
+ * Add steps to compute formatted_expr, pathspec, and
+ * PASSING arg expressions as things that must be
+ * evaluated before the "main" JSON path evaluation.
+ */
ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
- &jsestate->formatted_expr->value,
- &jsestate->formatted_expr->isnull);
-
- jsestate->pathspec =
- palloc(sizeof(*jsestate->pathspec));
-
+ &pre_eval->formatted_expr.value,
+ &pre_eval->formatted_expr.isnull);
ExecInitExprRec((Expr *) jexpr->path_spec, state,
- &jsestate->pathspec->value,
- &jsestate->pathspec->isnull);
-
- jsestate->res_expr =
- palloc(sizeof(*jsestate->res_expr));
-
- jsestate->result_expr = jexpr->result_coercion
- ? ExecInitExprWithCaseValue((Expr *) jexpr->result_coercion->expr,
- state->parent,
- &jsestate->res_expr->value,
- &jsestate->res_expr->isnull)
- : NULL;
-
- jsestate->default_on_empty = !jexpr->on_empty ? NULL :
- ExecInitExpr((Expr *) jexpr->on_empty->default_expr,
- state->parent);
-
- jsestate->default_on_error =
- ExecInitExpr((Expr *) jexpr->on_error->default_expr,
- state->parent);
-
- if (jexpr->omit_quotes ||
- (jexpr->result_coercion && jexpr->result_coercion->via_io))
- {
- Oid typinput;
- FmgrInfo *finfo;
-
- /* lookup the result type's input function */
- getTypeInputInfo(jexpr->returning->typid, &typinput,
- &jsestate->input.typioparam);
- finfo = palloc0(sizeof(FmgrInfo));
- fmgr_info(typinput, finfo);
- jsestate->input.finfo = finfo;
- }
-
- jsestate->args = NIL;
-
+ &pre_eval->pathspec.value,
+ &pre_eval->pathspec.isnull);
+
+ /*
+ * However, before pushing steps for PASSING args, push a step
+ * that will decide whether to skip evaluating the args and
+ * JSON path evaluation depending on whether either of
+ * formatted_expr and pathspec is NULL.
+ */
+ scratch.opcode = EEOP_JSONEXPR_SKIP;
+ scratch.d.jsonexpr_skip.pre_eval = pre_eval;
+ scratch.d.jsonexpr_skip.post_eval = post_eval;
+ skip_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ jsestate->pre_eval.args = NIL;
forboth(argexprlc, jexpr->passing_values,
argnamelc, jexpr->passing_names)
{
@@ -2628,19 +2613,202 @@ ExecInitExprRec(Expr *node, ExprState *state,
var->name = pstrdup(argname->sval);
var->typid = exprType((Node *) argexpr);
var->typmod = exprTypmod((Node *) argexpr);
- var->estate = ExecInitExpr(argexpr, state->parent);
+
+ /*
+ * Not necessary when being evaluated for a JsonExpr, like
+ * in this case.
+ */
+ var->estate = NULL;
var->econtext = NULL;
var->mcxt = NULL;
- var->evaluated = false;
- var->value = (Datum) 0;
- var->isnull = true;
- jsestate->args =
- lappend(jsestate->args, var);
+ /*
+ * Mark these as always evaluated because they must have
+ * been evaluated before JSON path evaluation begins,
+ * because we haven't pushed the step for the latter yet.
+ */
+ var->evaluated = true;
+
+ ExecInitExprRec((Expr *) argexpr, state,
+ &var->value,
+ &var->isnull);
+
+ pre_eval->args = lappend(pre_eval->args, var);
+ }
+
+ /* Now push the step for the actual JSON path evaluation. */
+ scratch.opcode = EEOP_JSONEXPR_PATH;
+ scratch.d.jsonexpr.jsestate = jsestate;
+ ExprEvalPushStep(state, &scratch);
+
+ /*
+ * Now push steps to control the expressions evaluated based
+ * on the result of JSON path evaluation.
+ */
+
+ /*
+ * Step to handle ON EMPTY behavior correctly. Might be
+ * skipped over (along with default expression steps added
+ * below) if 1) the JSON path evaluation result is not empty,
+ * 2) ON ERROR behavior must be applied instead. Jump target
+ * addressesnecessary to do so will be set later.
+ */
+ onempty_step_off = -1;
+ if (jexpr->on_empty)
+ {
+ scratch.opcode = EEOP_JSONEXPR_ONEMPTY;
+ scratch.d.jsonexpr_onempty.post_eval = post_eval;
+ scratch.d.jsonexpr_onempty.jexpr = jexpr;
+ onempty_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Push step(s) for the default ON EMPTY expression. */
+ if (jexpr->on_empty->default_expr)
+ ExecInitExprRec((Expr *) jexpr->on_empty->default_expr,
+ state, resv, resnull);
+ }
+
+ /*
+ * Step to handle ON ERROR behavior correctly. Might be
+ * skipped over (along with default expression steps added
+ * below) if no error occurs. Jump target address necessary
+ * to do so will be set later.
+ */
+ Assert(jexpr->on_error != NULL);
+ scratch.opcode = EEOP_JSONEXPR_ONERROR;
+ scratch.d.jsonexpr_onerror.post_eval = post_eval;
+ scratch.d.jsonexpr_onerror.jexpr = jexpr;
+ onerror_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Push step(s) for the default ON ERROR expression. */
+ if (jexpr->on_error->default_expr)
+ ExecInitExprRec((Expr *) jexpr->on_error->default_expr,
+ state, resv, resnull);
+
+ /*
+ * Step to handle applying coercion correctly. Might be
+ * skipped if no coercion needs to be applied separately
+ * from JSON path evaluation itself. Jump target address
+ * necessary to do so will be set later.
+ */
+ scratch.opcode = EEOP_JSONEXPR_COERCION;
+ scratch.d.jsonexpr_coercion.jsestate = jsestate;
+ scratch.d.jsonexpr_coercion.jump_skip_coercion = state->steps_len + 1;
+ coercion_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ /*
+ * Step to apply ON ERROR behavior if an error occurs when
+ * applying coercion. Will be skipped over (along with default
+ * expression steps added) if no error occurs. Jump target
+ * address necessary to do so will be set later.
+ */
+ scratch.opcode = EEOP_JSONEXPR_COERCION_ERROR;
+ scratch.d.jsonexpr_coercion_error.post_eval = post_eval;
+ scratch.d.jsonexpr_coercion_error.jexpr = jexpr;
+ coercion_error_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Push steps to evaluate the default ON ERROR expression. */
+ if (jexpr->on_error->default_expr)
+ ExecInitExprRec((Expr *) jexpr->on_error->default_expr,
+ state, resv, resnull);
+
+ /*
+ * Step to handle applying result_coercion if needed. Might
+ * be skipped if computing result_coercion is rendered
+ * unnecessary by earlier steps. Jump target address to do
+ * so will be set later.
+ */
+ result_step_off = state->steps_len;
+ if (jexpr->result_coercion && jexpr->result_coercion->expr)
+ {
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
+
+ scratch.opcode = EEOP_JSONEXPR_RESULT;
+ scratch.d.jsonexpr_result.post_eval = post_eval;
+ scratch.d.jsonexpr_result.jexpr = jexpr;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Steps to compute result_coercion expression. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
+
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) jexpr->result_coercion->expr,
+ state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
+ }
+
+ /*
+ * Adjust jump target addresses in various post-eval steps now
+ * that we have all the steps in place.
+ */
+
+ /* EEOP_JSONEXPR_SKIP */
+ as = &state->steps[skip_step_off];
+ as->d.jsonexpr_skip.jump_coercion = coercion_step_off;
+
+ /* EEOP_JSONEXPR_ONEMPTY */
+ if (onempty_step_off >= 0)
+ {
+ as = &state->steps[onempty_step_off];
+ as->d.jsonexpr_onempty.jump_error = onerror_step_off;
+ as->d.jsonexpr_onempty.jump_coercion = coercion_step_off;
}
- jsestate->cache = NULL;
+ /* EEOP_JSONEXPR_ONERROR */
+ as = &state->steps[onerror_step_off];
+ as->d.jsonexpr_onerror.jump_coercion = coercion_step_off;
+
+ /* EEOP_JSONEXPR_COERCION */
+ as = &state->steps[coercion_step_off];
+ as->d.jsonexpr_coercion.jump_coercion_error = coercion_error_step_off;
+
+ /* EEOP_JSONEXPR_COERCION_ERROR */
+ as = &state->steps[coercion_error_step_off];
+ as->d.jsonexpr_coercion_error.jump_coercion = coercion_step_off;
+ as->d.jsonexpr_coercion_error.jump_skip_coercion_error = result_step_off;
+
+ /* EEOP_JSONEXPR_RESULT */
+ as = &state->steps[result_step_off];
+ as->d.jsonexpr_result.jump_skip_result = state->steps_len;
+
+ /*
+ * Set if we must use a sub-transaction around path evaluation
+ * that can be aborted on error instead of aborting the parent
+ * query.
+ */
+ jsestate->needSubtrans =
+ ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ jsestate->coercionNeedSubtrans =
+ (!jsestate->needSubtrans && !throwErrors);
+
+ /*
+ * Set RETURNING type's input function used by
+ * ExecEvalJsonExprCoercion().
+ */
+ if (jexpr->omit_quotes ||
+ (jexpr->result_coercion && jexpr->result_coercion->via_io))
+ {
+ Oid typinput;
+ FmgrInfo *finfo;
+
+ /* lookup the result type's input function */
+ getTypeInputInfo(jexpr->returning->typid, &typinput,
+ &jsestate->input.typioparam);
+ finfo = palloc0(sizeof(FmgrInfo));
+ fmgr_info(typinput, finfo);
+ jsestate->input.finfo = finfo;
+ }
+ /* Set up coercion related data structures. */
if (jexpr->coercions)
{
JsonCoercion **coercion;
@@ -2648,11 +2816,8 @@ ExecInitExprRec(Expr *node, ExprState *state,
Datum *caseval;
bool *casenull;
- jsestate->coercion_expr =
- palloc(sizeof(*jsestate->coercion_expr));
-
- caseval = &jsestate->coercion_expr->value;
- casenull = &jsestate->coercion_expr->isnull;
+ caseval = resv;
+ casenull = resnull;
for (cstate = &jsestate->coercions.null,
coercion = &jexpr->coercions->null;
@@ -2667,7 +2832,6 @@ ExecInitExprRec(Expr *node, ExprState *state,
}
}
- ExprEvalPushStep(state, &scratch);
break;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 0512a81c7c..7a533c16df 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -158,6 +158,15 @@ static TupleDesc get_cached_rowtype(Oid type_id, int32 typmod,
bool *changed);
static void ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, bool checkisnull);
+static Datum ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null);
+typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
+ Datum item, bool *resnull, void *p, bool *error);
+static Datum ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
+ ExprContext *econtext,
+ Datum res, bool *resnull,
+ void *p, bool *error, bool subtrans);
+static Datum ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
+ Datum res, bool *isNull, void *p, bool *error);
/* fast-path evaluation functions */
static Datum ExecJustInnerVar(ExprState *state, ExprContext *econtext, bool *isnull);
@@ -490,7 +499,13 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SUBPLAN,
&&CASE_EEOP_JSON_CONSTRUCTOR,
&&CASE_EEOP_IS_JSON,
- &&CASE_EEOP_JSONEXPR,
+ &&CASE_EEOP_JSONEXPR_SKIP,
+ &&CASE_EEOP_JSONEXPR_PATH,
+ &&CASE_EEOP_JSONEXPR_ONERROR,
+ &&CASE_EEOP_JSONEXPR_ONEMPTY,
+ &&CASE_EEOP_JSONEXPR_COERCION,
+ &&CASE_EEOP_JSONEXPR_COERCION_ERROR,
+ &&CASE_EEOP_JSONEXPR_RESULT,
&&CASE_EEOP_AGG_STRICT_DESERIALIZE,
&&CASE_EEOP_AGG_DESERIALIZE,
&&CASE_EEOP_AGG_STRICT_INPUT_CHECK_ARGS,
@@ -1817,13 +1832,178 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
- EEO_CASE(EEOP_JSONEXPR)
+ EEO_CASE(EEOP_JSONEXPR_SKIP)
+ {
+ JsonExprPreEvalState *pre_eval = op->d.jsonexpr_skip.pre_eval;
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_skip.post_eval;
+
+ /*
+ * Skip JSON evaluation if either of the input expressions has
+ * turned out to be NULL, though do execute domain checks for
+ * NULLs, which are handled by the coercion step.
+ */
+ if (pre_eval->formatted_expr.isnull || pre_eval->pathspec.isnull)
+ {
+ post_eval->coercion = NULL;
+ *op->resvalue = 0;
+ *op->resnull = true;
+ EEO_JUMP(op->d.jsonexpr_skip.jump_coercion);
+ }
+
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_PATH)
{
/* too complex for an inline implementation */
ExecEvalJson(state, op, econtext);
EEO_NEXT();
}
+ EEO_CASE(EEOP_JSONEXPR_ONEMPTY)
+ {
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_onempty.post_eval;
+ JsonExpr *jexpr = op->d.jsonexpr_onempty.jexpr;
+
+ /*
+ * Skip ON EMPTY handling if not empty or if it must be handled
+ * with a ON ERROR behavior instead.
+ */
+ if (post_eval->error)
+ EEO_JUMP(op->d.jsonexpr_onempty.jump_error);
+ else if (!post_eval->empty)
+ EEO_JUMP(op->d.jsonexpr_onempty.jump_coercion);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate
+ * value and skip over to the coercion step.
+ */
+ if (jexpr->on_empty->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ *op->resvalue = ExecEvalJsonBehavior(jexpr->on_empty,
+ op->resnull);
+ EEO_JUMP(op->d.jsonexpr_onempty.jump_coercion);
+ }
+
+ /*
+ * Evaluate the default ON EMPTY expression.
+ *
+ * Nothing for _COERCION and _RESULT steps to do as the
+ * expression we are about to compute has been coerced
+ * appropriately by the parser.
+ */
+ post_eval->coercion_done = true;
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_ONERROR)
+ {
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_onerror.post_eval;
+ JsonExpr *jexpr = op->d.jsonexpr_onerror.jexpr;
+
+ /* Nothing to do if no error has been found to begin with. */
+ if (!post_eval->error)
+ EEO_JUMP(op->d.jsonexpr_onerror.jump_coercion);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate
+ * value and skip over to the coercion step.
+ */
+ if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ /* Execute non-default ON ERROR behavior */
+ *op->resvalue = ExecEvalJsonBehavior(jexpr->on_error,
+ op->resnull);
+ EEO_JUMP(op->d.jsonexpr_onerror.jump_coercion);
+ }
+
+ /*
+ * Evaluate the default ON ERROR expression. See the note
+ * above regarding coercion.
+ */
+ post_eval->coercion_done = true;
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_COERCION_ERROR)
+ {
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_coercion_error.post_eval;
+ JsonExpr *jexpr = op->d.jsonexpr_coercion_error.jexpr;
+
+ /* Nothing to if no error was found when applying coercion. */
+ if (!post_eval->coercion_error)
+ EEO_JUMP(op->d.jsonexpr_coercion_error.jump_skip_coercion_error);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate
+ * value and skip over to the coercion step.
+ */
+ if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ /* Execute non-default ON ERROR behavior */
+ *op->resvalue = ExecEvalJsonBehavior(jexpr->on_error,
+ op->resnull);
+
+ /* Coercion the ON ERROR expression. */
+ post_eval->coercion_done = false;
+ post_eval->coercion = NULL;
+ EEO_JUMP(op->d.jsonexpr_coercion_error.jump_coercion);
+ }
+
+ /*
+ * Evaluate the default ON ERROR expression. See the note
+ * above regarding coercion.
+ */
+ post_eval->coercion_done = true;
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_COERCION)
+ {
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ bool coersionUseSubtrans;
+
+ /*
+ * Don't use a sub-transaction if coercing the ON ERROR expression
+ * after the previous coercion attempt caused an error so that any
+ * that any errors that are encountered downstream this time are
+ * thrown there, that is, not rethrown to be handled here.
+ * XXX - doesn't that violate ON ERROR behavior? Actually, that is
+ * how it behaved even before this expression eval logic rewrite!
+ */
+ coersionUseSubtrans =
+ (jsestate->coercionNeedSubtrans &&
+ !post_eval->coercion_error);
+
+ /* ExecEvalJson() may have found coercion to be unnecessary. */
+ if (post_eval->coercion_done)
+ EEO_JUMP(op->d.jsonexpr_coercion.jump_skip_coercion);
+
+ post_eval->coercion_error = false;
+ *op->resvalue =
+ ExecEvalJsonExprSubtrans(ExecEvalJsonExprCoercion, op, econtext,
+ *op->resvalue,
+ op->resnull, NULL,
+ &post_eval->coercion_error,
+ coersionUseSubtrans);
+ if (post_eval->coercion_error)
+ EEO_JUMP(op->d.jsonexpr_coercion.jump_coercion_error);
+
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_RESULT)
+ {
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_result.post_eval;
+
+ if (post_eval->coercion_done)
+ EEO_JUMP(op->d.jsonexpr_result.jump_skip_result);
+
+ /* Evaluate result_coercion expression. */
+ EEO_NEXT();
+ }
+
EEO_CASE(EEOP_LAST)
{
/* unreachable */
@@ -4602,8 +4782,7 @@ ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
* Evaluate a JSON error/empty behavior result.
*/
static Datum
-ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
- ExprState *default_estate, bool *is_null)
+ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null)
{
*is_null = false;
@@ -4628,7 +4807,9 @@ ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
return (Datum) 0;
case JSON_BEHAVIOR_DEFAULT:
- return ExecEvalExpr(default_estate, econtext, is_null);
+ /* Always handled in the caller. */
+ Assert(false);
+ return (Datum) 0;
default:
elog(ERROR, "unrecognized SQL/JSON behavior %d", behavior->btype);
@@ -4643,18 +4824,25 @@ static Datum
ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
Datum res, bool *isNull, void *p, bool *error)
{
- ExprState *estate = p;
- JsonExprState *jsestate;
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ JsonExpr *jexpr = jsestate->jsexpr;
+ ExprState *estate = post_eval->coercion ?
+ post_eval->coercion->estate : NULL;
+
+ /*
+ * Initially assume we will indeed perform the coercion either
+ * using post_eval->coercion, with the input function for the
+ * output type, or with json_populate_type().
+ */
+ post_eval->coercion_done = true;
if (estate) /* coerce using specified expression */
return ExecEvalExpr(estate, econtext, isNull);
- jsestate = op->d.jsonexpr.jsestate;
-
- if (jsestate->jsexpr->op != JSON_EXISTS_OP)
+ if (jexpr->op != JSON_EXISTS_OP)
{
- JsonCoercion *coercion = jsestate->jsexpr->result_coercion;
- JsonExpr *jexpr = jsestate->jsexpr;
+ JsonCoercion *coercion = jexpr->result_coercion;
Jsonb *jb = *isNull ? NULL : DatumGetJsonbP(res);
if ((coercion && coercion->via_io) ||
@@ -4672,18 +4860,16 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
return json_populate_type(res, JSONBOID,
jexpr->returning->typid,
jexpr->returning->typmod,
- &jsestate->cache,
+ &post_eval->cache,
econtext->ecxt_per_query_memory,
isNull);
}
- if (jsestate->result_expr)
- {
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *isNull;
-
- res = ExecEvalExpr(jsestate->result_expr, econtext, isNull);
- }
+ /*
+ * Let the caller know that no coercion was done here, so it can
+ * coerce with jexpr->result_coercion if there's one.
+ */
+ post_eval->coercion_done = false;
return res;
}
@@ -4717,17 +4903,30 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
if (!var)
return -1;
- if (!var->evaluated)
+ /*
+ * When belonging to a JsonExpr, path variables are computed with the
+ * JsonExpr's ExprState (var->estate is NULL), so don't need to be computed
+ * here. In some other cases, such as when the path variables belonging
+ * to a JsonTable instead, those variables must be evaluated on their own,
+ * without the enclosing JsonExpr itself needing to be evaluated, so must
+ * be handled here.
+ */
+ if (var->estate && !var->evaluated)
{
MemoryContext oldcxt = var->mcxt ?
MemoryContextSwitchTo(var->mcxt) : NULL;
+ Assert(var->econtext != NULL);
var->value = ExecEvalExpr(var->estate, var->econtext, &var->isnull);
var->evaluated = true;
if (oldcxt)
MemoryContextSwitchTo(oldcxt);
}
+ else
+ {
+ Assert(var->evaluated);
+ }
if (var->isnull)
{
@@ -4832,9 +5031,6 @@ ExecPrepareJsonItemCoercion(JsonbValue *item,
return res;
}
-typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
- Datum item, bool *resnull, void *p, bool *error);
-
static Datum
ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
ExprContext *econtext,
@@ -4905,7 +5101,6 @@ typedef struct
{
JsonPath *path;
bool *error;
- bool coercionInSubtrans;
} ExecEvalJsonExprContext;
static Datum
@@ -4916,16 +5111,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
ExecEvalJsonExprContext *cxt = pcxt;
JsonPath *path = cxt->path;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
- ExprState *estate = NULL;
- bool empty = false;
+ bool *empty = &post_eval->empty;
Datum res = (Datum) 0;
switch (jexpr->op)
{
case JSON_QUERY_OP:
- res = JsonPathQuery(item, path, jexpr->wrapper, &empty, error,
- jsestate->args);
+ res = JsonPathQuery(item, path, jexpr->wrapper, empty, error,
+ pre_eval->args);
if (error && *error)
{
*resnull = true;
@@ -4937,16 +5133,19 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
case JSON_VALUE_OP:
{
struct JsonCoercionState *jcstate;
- JsonbValue *jbv = JsonPathValue(item, path, &empty, error,
- jsestate->args);
+ JsonbValue *jbv = JsonPathValue(item, path, empty, error,
+ pre_eval->args);
if (error && *error)
+ {
+ *resnull = true;
return (Datum) 0;
+ }
if (!jbv) /* NULL or empty */
break;
- Assert(!empty);
+ Assert(!*empty);
*resnull = false;
@@ -4964,7 +5163,6 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
jsestate->jsexpr->returning,
&jsestate->coercions,
&jcstate);
-
if (jcstate->coercion &&
(jcstate->coercion->via_io ||
jcstate->coercion->via_populate))
@@ -4972,6 +5170,7 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
if (error)
{
*error = true;
+ *resnull = true;
return (Datum) 0;
}
@@ -4983,32 +5182,30 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_SQL_JSON_ITEM_CANNOT_BE_CAST_TO_TARGET_TYPE),
errmsg("SQL/JSON item cannot be cast to target type")));
}
- else if (!jcstate->estate)
- return res; /* no coercion */
-
- /* coerce using specific expression */
- estate = jcstate->estate;
- jsestate->coercion_expr->value = res;
- jsestate->coercion_expr->isnull = *resnull;
+ else if (jcstate->estate == NULL)
+ {
+ /* No coercion needed */
+ post_eval->coercion_done = true;
+ }
+ else
+ {
+ /* Coerce using specific expression */
+ post_eval->coercion = jcstate;
+ }
break;
}
case JSON_EXISTS_OP:
{
bool exists = JsonPathExists(item, path,
- jsestate->args,
+ pre_eval->args,
error);
*resnull = error && *error;
res = BoolGetDatum(exists);
- if (!jsestate->result_expr)
+ if (jexpr->result_coercion == NULL)
return res;
-
- /* coerce using result expression */
- estate = jsestate->result_expr;
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *resnull;
break;
}
@@ -5021,7 +5218,11 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
return (Datum) 0;
}
- if (empty)
+ /*
+ * If the ON EMPTY behavior is to cause an error, do so here. Other
+ * behaviors will be handled by the caller.
+ */
+ if (*empty)
{
Assert(jexpr->on_empty); /* it is not JSON_EXISTS */
@@ -5037,24 +5238,9 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_NO_SQL_JSON_ITEM),
errmsg("no SQL/JSON item")));
}
-
- if (jexpr->on_empty->btype == JSON_BEHAVIOR_DEFAULT)
-
- /*
- * Execute DEFAULT expression as a coercion expression, because
- * its result is already coerced to the target type.
- */
- estate = jsestate->default_on_empty;
- else
- /* Execute ON EMPTY behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_empty,
- jsestate->default_on_empty,
- resnull);
}
- return ExecEvalJsonExprSubtrans(ExecEvalJsonExprCoercion, op, econtext,
- res, resnull, estate, error,
- cxt->coercionInSubtrans);
+ return res;
}
bool
@@ -5082,64 +5268,30 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
{
ExecEvalJsonExprContext cxt;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
Datum item;
Datum res = (Datum) 0;
JsonPath *path;
- ListCell *lc;
- bool error = false;
- bool needSubtrans;
bool throwErrors = jexpr->on_error->btype == JSON_BEHAVIOR_ERROR;
*op->resnull = true; /* until we get a result */
*op->resvalue = (Datum) 0;
- if (jsestate->formatted_expr->isnull || jsestate->pathspec->isnull)
- {
- /* execute domain checks for NULLs */
- (void) ExecEvalJsonExprCoercion(op, econtext, res, op->resnull,
- NULL, NULL);
+ item = pre_eval->formatted_expr.value;
+ path = DatumGetJsonPathP(pre_eval->pathspec.value);
- Assert(*op->resnull);
- return;
- }
-
- item = jsestate->formatted_expr->value;
- path = DatumGetJsonPathP(jsestate->pathspec->value);
-
- /* reset JSON path variable contexts */
- foreach(lc, jsestate->args)
- {
- JsonPathVariableEvalContext *var = lfirst(lc);
-
- var->econtext = econtext;
- var->evaluated = false;
- }
-
- needSubtrans = ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ /* Reset JsonExprPostEvalState for this evaluation. */
+ memset(post_eval, 0, sizeof(*post_eval));
cxt.path = path;
- cxt.error = throwErrors ? NULL : &error;
- cxt.coercionInSubtrans = !needSubtrans && !throwErrors;
- Assert(!needSubtrans || cxt.error);
+ cxt.error = throwErrors ? NULL : &post_eval->error;
+ Assert(!jsestate->needSubtrans || cxt.error);
res = ExecEvalJsonExprSubtrans(ExecEvalJsonExpr, op, econtext, item,
op->resnull, &cxt, cxt.error,
- needSubtrans);
-
- if (error)
- {
- /* Execute ON ERROR behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_error,
- jsestate->default_on_error,
- op->resnull);
-
- /* result is already coerced in DEFAULT behavior case */
- if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
- res = ExecEvalJsonExprCoercion(op, econtext, res,
- op->resnull,
- NULL, NULL);
- }
+ jsestate->needSubtrans);
*op->resvalue = res;
}
diff --git a/src/backend/jit/llvm/llvmjit_expr.c b/src/backend/jit/llvm/llvmjit_expr.c
index b6b6512ef1..c133600005 100644
--- a/src/backend/jit/llvm/llvmjit_expr.c
+++ b/src/backend/jit/llvm/llvmjit_expr.c
@@ -2359,11 +2359,44 @@ llvm_compile_expr(ExprState *state)
LLVMBuildBr(b, opblocks[opno + 1]);
break;
- case EEOP_JSONEXPR:
+ case EEOP_JSONEXPR_SKIP:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_PATH:
build_EvalXFunc(b, mod, "ExecEvalJson",
v_state, op, v_econtext);
LLVMBuildBr(b, opblocks[opno + 1]);
break;
+ case EEOP_JSONEXPR_ONERROR:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_ONEMPTY:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_COERCION:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_COERCION_ERROR:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_RESULT:
+ {
+ /*
+ * Perform the step at jsonexpr_result.jumpnext if the
+ * coercion already done, which effectively skips the
+ * step(s) to perform the coercion using the result
+ * expression.
+ * XXX - Hoping this DTRT.
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b, LLVMIntEQ,
+ l_sbool_const(op->d.jsonexpr_result.post_eval->coercion_done),
+ l_sbool_const(1),
+ ""),
+ opblocks[op->d.jsonexpr_result.jump_skip_result],
+ opblocks[opno + 1]);
+ }
+ break;
case EEOP_LAST:
Assert(false);
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index e55a572854..358b831f7a 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -244,7 +244,13 @@ typedef enum ExprEvalOp
EEOP_SUBPLAN,
EEOP_JSON_CONSTRUCTOR,
EEOP_IS_JSON,
- EEOP_JSONEXPR,
+ EEOP_JSONEXPR_SKIP,
+ EEOP_JSONEXPR_PATH,
+ EEOP_JSONEXPR_ONERROR,
+ EEOP_JSONEXPR_ONEMPTY,
+ EEOP_JSONEXPR_COERCION,
+ EEOP_JSONEXPR_COERCION_ERROR,
+ EEOP_JSONEXPR_RESULT,
/* aggregation related nodes */
EEOP_AGG_STRICT_DESERIALIZE,
@@ -682,12 +688,71 @@ typedef struct ExprEvalStep
JsonIsPredicate *pred; /* original expression node */
} is_json;
- /* for EEOP_JSONEXPR */
+ /* for EEOP_JSONEXPR_PATH */
struct
{
struct JsonExprState *jsestate;
} jsonexpr;
+ /* for EEOP_JSONEXPR_SKIP */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprPreEvalState *pre_eval;
+ struct JsonExprPostEvalState *post_eval;
+ int jump_coercion;
+ } jsonexpr_skip;
+
+ /* for EEOP_JSONEXPR_ONEMPTY */
+ struct
+ {
+ /* Points to jsonexpr.jsestate->post_eval */
+ struct JsonExprPostEvalState *post_eval;
+ /* Same as jsonexpr.jsestate->jsexpr */
+ JsonExpr *jexpr;
+ int jump_error;
+ int jump_coercion;
+ } jsonexpr_onempty;
+
+ /* for EEOP_JSONEXPR_ONERROR */
+ struct
+ {
+ /* Points to jsonexpr.jsestate->post_eval */
+ struct JsonExprPostEvalState *post_eval;
+ /* Same as jsonexpr.jsestate->jsexpr */
+ JsonExpr *jexpr;
+ int jump_coercion;
+ } jsonexpr_onerror;
+
+ /* for EEOP_JSONEXPR_COERCION */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_skip_coercion;
+ int jump_coercion_error;
+ } jsonexpr_coercion;
+
+ /* for EEOP_JSONEXPR_COERCION_ERROR */
+ struct
+ {
+ /* Points to jsonexpr.jsestate->post_eval */
+ struct JsonExprPostEvalState *post_eval;
+ /* Same as jsonexpr.jsestate->jsexpr */
+ JsonExpr *jexpr;
+ int jump_skip_coercion_error;
+ int jump_coercion;
+ } jsonexpr_coercion_error;
+
+ /* for EEOP_JSONEXPR_RESULT */
+ struct
+ {
+ /* Points to jsonexpr.jsestate->post_eval */
+ struct JsonExprPostEvalState *post_eval;
+ /* Same as jsonexpr.jsestate->jsexpr */
+ JsonExpr *jexpr;
+ int jump_skip_result;
+ } jsonexpr_result;
} d;
} ExprEvalStep;
@@ -747,30 +812,69 @@ typedef struct JsonConstructorExprState
int nargs;
} JsonConstructorExprState;
+/*
+ * State for (EEOP_JSONEXPR_) ONERROR, ONEMPTY, COERCION, COERCION_ERROR,
+ * and RESULT steps that will run after EEOP_JSONEXPR_PATH.
+ */
+typedef struct JsonExprPostEvalState
+{
+ /* Is JSON item empty? */
+ bool empty;
+
+ /* Did JSON item evaluation cause an error? */
+ bool error;
+
+ /* Did coercion evaluation cause an error? */
+ bool coercion_error;
+
+ /* Has the result been coerced properly? */
+ bool coercion_done;
+
+ /* Cache for json_populate_type() called for coercion in some cases */
+ void *cache;
+
+ /*
+ * Coercion to be applied to JSON item returned by JsonPathValue(),
+ * set by ExecPrepareJsonItemCoercion(). */
+ struct JsonCoercionState *coercion;
+} JsonExprPostEvalState;
+
+/*
+ * Information computed before evaluating a JsonExpr expression.
+ */
+typedef struct JsonExprPreEvalState
+{
+ /* value/isnull for JsonExpr.formatted_expr */
+ NullableDatum formatted_expr;
+
+ /* value/isnull for JsonExpr.pathspec */
+ NullableDatum pathspec;
+
+ /* JsonPathVariableEvalContext entries for JsonExpr.passing_values */
+ List *args;
+} JsonExprPreEvalState;
+
/* EEOP_JSONEXPR state, too big to inline */
typedef struct JsonExprState
{
JsonExpr *jsexpr; /* original expression node */
+ JsonExprPreEvalState pre_eval;
+ JsonExprPostEvalState post_eval;
+
+ /*
+ * Should use a sub-transaction for path evaluation and subsequent
+ * coercion evaluation, if any?
+ */
+ bool needSubtrans;
+ bool coercionNeedSubtrans;
+
struct
{
FmgrInfo *finfo; /* typinput function for output type */
Oid typioparam;
} input; /* I/O info for output type */
- NullableDatum
- *formatted_expr, /* formatted context item value */
- *res_expr, /* result item */
- *coercion_expr, /* input for JSON item coercion */
- *pathspec; /* path specification value */
-
- ExprState *result_expr; /* coerced to output type */
- ExprState *default_on_empty; /* ON EMPTY DEFAULT expression */
- ExprState *default_on_error; /* ON ERROR DEFAULT expression */
- List *args; /* passing arguments */
-
- void *cache; /* cache for json_populate_type() */
-
struct JsonCoercionsState
{
struct JsonCoercionState
@@ -779,7 +883,7 @@ typedef struct JsonExprState
ExprState *estate; /* coercion expression state */
} null,
string,
- numeric ,
+ numeric,
boolean,
date,
time,
--
2.34.1
0003-Use-one-ExprState-to-implement-JsonItemCoercions.patch (text/x-patch, 17.1K) download
From 5d904e7f8357345079531251849c9e4fe9d161b1 Mon Sep 17 00:00:00 2001
From: amitlan <amitlangote09@gmail.com>
Date: Mon, 18 Jul 2022 11:06:04 -0400
Subject: [PATCH 3/3] Use one ExprState to implement JsonItemCoercions
Current implementation assigns one ExprState for every JsonCoercion
member of JsonItemCoercions, which seems wasteful.
This commit instead assigns the ExprState to JsonItemCoercions itself
and arranges to compute the member JsonCoercion's expressions using
ExprEvalSteps instead.
---
src/backend/executor/execExpr.c | 113 ++++++++++++++++++----
src/backend/executor/execExprInterp.c | 133 ++++++++++++++++++++------
src/backend/jit/llvm/llvmjit_expr.c | 3 +
src/backend/optimizer/util/clauses.c | 6 +-
src/include/executor/execExpr.h | 75 +++++++++------
src/include/utils/jsonb.h | 2 +
6 files changed, 256 insertions(+), 76 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 4dbb24aaee..be98544e05 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2786,7 +2786,8 @@ ExecInitExprRec(Expr *node, ExprState *state,
* query.
*/
jsestate->needSubtrans =
- ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ ExecEvalJsonNeedsSubTransaction(jexpr,
+ true /* coerce */);
jsestate->coercionNeedSubtrans =
(!jsestate->needSubtrans && !throwErrors);
@@ -2810,26 +2811,102 @@ ExecInitExprRec(Expr *node, ExprState *state,
/* Set up coercion related data structures. */
if (jexpr->coercions)
+ jsestate->coercions =
+ ExecInitExprWithCaseValue((Expr *) jexpr->coercions,
+ state->parent,
+ resv, resnull);
+
+ break;
+ }
+
+ case T_JsonItemCoercions:
+ {
+ JsonItemCoercions *coercions = castNode(JsonItemCoercions,
+ node);
+ JsonItemCoercionsState *jcstate =
+ palloc0(sizeof(JsonItemCoercionsState));
+ JsonCoercion **coercion;
+ JsonItemCoercionState *cstate;
+ ExprEvalStep *as;
+ int json_item_coercion_step_id;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
+
+ /*
+ * First push a step to read the value provided by the parent
+ * JsonExpr via a CaseTestExpr.
+ */
+ scratch.opcode = EEOP_CASE_TESTVAL;
+ scratch.d.casetest.value = state->innermost_caseval;
+ scratch.d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Push the control step. */
+ scratch.opcode = EEOP_JSON_ITEM_COERCION;
+ scratch.d.json_item_coercion.jcstate = jcstate;
+ json_item_coercion_step_id = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+ /* Will set jump_skip_coercion target address below. */
+
+ /*
+ * Now push the steps of individual coercion's expression, if
+ * needed.
+ */
+ for (cstate = &jcstate->null,
+ coercion = &coercions->null;
+ coercion <= &coercions->composite;
+ coercion++, cstate++)
{
- JsonCoercion **coercion;
- struct JsonCoercionState *cstate;
- Datum *caseval;
- bool *casenull;
-
- caseval = resv;
- casenull = resnull;
-
- for (cstate = &jsestate->coercions.null,
- coercion = &jexpr->coercions->null;
- coercion <= &jexpr->coercions->composite;
- coercion++, cstate++)
+ cstate->coercion = *coercion;
+ if (cstate->coercion && cstate->coercion->expr)
{
- cstate->coercion = *coercion;
- cstate->estate = *coercion ?
- ExecInitExprWithCaseValue((Expr *) (*coercion)->expr,
- state->parent,
- caseval, casenull) : NULL;
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
+
+ cstate->jump_eval_expr = state->steps_len;
+
+ /* Push step(s) to compute (*coercion)->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
+
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) cstate->coercion->expr,
+ state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
}
+ else
+ cstate->jump_eval_expr = -1;
+
+ /* Emit JUMP step to jump to end of coercions code */
+ scratch.opcode = EEOP_JUMP;
+
+ /*
+ * Remember JUMP step address to set the actual jump
+ * target address below.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len);
+ ExprEvalPushStep(state, &scratch);
+ }
+
+ /*
+ * Adjust the jump target address of the control step and all
+ * the jumps we added in the above loop to make them point to
+ * the step after the last step that would have been added
+ * above.
+ */
+ as = &state->steps[json_item_coercion_step_id];
+ as->d.json_item_coercion.jump_skip_item_coercion = state->steps_len;
+ foreach(lc, adjust_jumps)
+ {
+ int jump_step_id = lfirst_int(lc);
+
+ as = &state->steps[jump_step_id];
+ as->d.jump.jumpdone = state->steps_len;
}
break;
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 7a533c16df..f09d643921 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -506,6 +506,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_JSONEXPR_COERCION,
&&CASE_EEOP_JSONEXPR_COERCION_ERROR,
&&CASE_EEOP_JSONEXPR_RESULT,
+ &&CASE_EEOP_JSON_ITEM_COERCION,
&&CASE_EEOP_AGG_STRICT_DESERIALIZE,
&&CASE_EEOP_AGG_DESERIALIZE,
&&CASE_EEOP_AGG_STRICT_INPUT_CHECK_ARGS,
@@ -1844,7 +1845,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
*/
if (pre_eval->formatted_expr.isnull || pre_eval->pathspec.isnull)
{
- post_eval->coercion = NULL;
+ post_eval->coercions = NULL;
*op->resvalue = 0;
*op->resnull = true;
EEO_JUMP(op->d.jsonexpr_skip.jump_coercion);
@@ -1946,7 +1947,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
/* Coercion the ON ERROR expression. */
post_eval->coercion_done = false;
- post_eval->coercion = NULL;
+ post_eval->coercions = NULL;
EEO_JUMP(op->d.jsonexpr_coercion_error.jump_coercion);
}
@@ -2004,6 +2005,23 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
+ EEO_CASE(EEOP_JSON_ITEM_COERCION)
+ {
+ JsonItemCoercionsState *jcstate = op->d.json_item_coercion.jcstate;
+ JsonItemCoercionState *cstate = NULL;
+
+ *op->resvalue =
+ ExecPrepareJsonItemCoercion(*op->resvalue, jcstate, &cstate);
+ if (cstate->coercion && cstate->coercion->expr)
+ {
+ Assert(cstate->jump_eval_expr >= 0);
+ EEO_JUMP(cstate->jump_eval_expr);
+ }
+
+ /* Skip over all of the steps added for this JsonItemCoercions. */
+ EEO_JUMP(op->d.json_item_coercion.jump_skip_item_coercion);
+ }
+
EEO_CASE(EEOP_LAST)
{
/* unreachable */
@@ -4827,8 +4845,7 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
- ExprState *estate = post_eval->coercion ?
- post_eval->coercion->estate : NULL;
+ ExprState *estate = post_eval->coercions;
/*
* Initially assume we will indeed perform the coercion either
@@ -4940,19 +4957,84 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
return id;
}
+/*
+ * Return a coercion among those given in 'coercions' for given
+ * JSON item.
+ */
+JsonCoercion *
+ExecGetJsonItemCoercion(JsonbValue *item, JsonItemCoercions *coercions)
+{
+ if (item->type == jbvBinary &&
+ JsonContainerIsScalar(item->val.binary.data))
+ {
+ JsonbValue buf;
+ bool res PG_USED_FOR_ASSERTS_ONLY;
+
+ res = JsonbExtractScalar(item->val.binary.data, &buf);
+ item = &buf;
+ Assert(res);
+ }
+
+ /* get coercion state reference and datum of the corresponding SQL type */
+ switch (item->type)
+ {
+ case jbvNull:
+ return coercions->null;
+
+ case jbvString:
+ return coercions->string;
+
+ case jbvNumeric:
+ return coercions->numeric;
+
+ case jbvBool:
+ return coercions->boolean;
+
+ case jbvDatetime:
+ switch (item->val.datetime.typid)
+ {
+ case DATEOID:
+ return coercions->date;
+ case TIMEOID:
+ return coercions->time;
+ case TIMETZOID:
+ return coercions->timetz;
+ case TIMESTAMPOID:
+ return coercions->timestamp;
+ case TIMESTAMPTZOID:
+ return coercions->timestamptz;
+ default:
+ elog(ERROR, "unexpected jsonb datetime type oid %u",
+ item->val.datetime.typid);
+ }
+ break;
+
+ case jbvArray:
+ case jbvObject:
+ case jbvBinary:
+ return coercions->composite;
+
+ default:
+ elog(ERROR, "unexpected jsonb value type %d", item->type);
+ }
+
+ Assert(false);
+ return NULL;
+}
+
/*
* Prepare SQL/JSON item coercion to the output type. Returned a datum of the
* corresponding SQL type and a pointer to the coercion state.
*/
Datum
-ExecPrepareJsonItemCoercion(JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pcoercion)
+ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pcoercion)
{
- struct JsonCoercionState *coercion;
+ JsonbValue *item = DatumGetJsonbValueP(itemval);
+ JsonItemCoercionState *coercion;
Datum res;
- JsonbValue buf;
+ JsonbValue buf;
if (item->type == jbvBinary &&
JsonContainerIsScalar(item->val.binary.data))
@@ -5132,7 +5214,7 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
case JSON_VALUE_OP:
{
- struct JsonCoercionState *jcstate;
+ JsonCoercion *coercion;
JsonbValue *jbv = JsonPathValue(item, path, empty, error,
pre_eval->args);
@@ -5158,14 +5240,12 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
break;
}
- /* Use coercion from SQL/JSON item type to the output type */
- res = ExecPrepareJsonItemCoercion(jbv,
- jsestate->jsexpr->returning,
- &jsestate->coercions,
- &jcstate);
- if (jcstate->coercion &&
- (jcstate->coercion->via_io ||
- jcstate->coercion->via_populate))
+ /*
+ * Error out no cast exists to coerce SQL/JSON item to the
+ * the output type
+ */
+ coercion = ExecGetJsonItemCoercion(jbv, jsestate->jsexpr->coercions);
+ if (coercion && (coercion->via_io || coercion->via_populate))
{
if (error)
{
@@ -5182,15 +5262,11 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_SQL_JSON_ITEM_CANNOT_BE_CAST_TO_TARGET_TYPE),
errmsg("SQL/JSON item cannot be cast to target type")));
}
- else if (jcstate->estate == NULL)
- {
- /* No coercion needed */
- post_eval->coercion_done = true;
- }
else
{
- /* Coerce using specific expression */
- post_eval->coercion = jcstate;
+ /* Coerce using specific expression. */
+ res = JsonbValuePGetDatum(jbv);
+ post_eval->coercions = jsestate->coercions;
}
break;
}
@@ -5244,8 +5320,7 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
}
bool
-ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *coercions)
+ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce)
{
if (jsexpr->on_error->btype == JSON_BEHAVIOR_ERROR)
return false;
@@ -5253,7 +5328,7 @@ ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
if (jsexpr->op == JSON_EXISTS_OP && !jsexpr->result_coercion)
return false;
- if (!coercions)
+ if (!coerce)
return true;
return false;
diff --git a/src/backend/jit/llvm/llvmjit_expr.c b/src/backend/jit/llvm/llvmjit_expr.c
index c133600005..5f06a87373 100644
--- a/src/backend/jit/llvm/llvmjit_expr.c
+++ b/src/backend/jit/llvm/llvmjit_expr.c
@@ -2397,6 +2397,9 @@ llvm_compile_expr(ExprState *state)
opblocks[opno + 1]);
}
break;
+ case EEOP_JSON_ITEM_COERCION:
+ /* XXX - handle! */
+ break;
case EEOP_LAST:
Assert(false);
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..9c02218155 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -901,7 +901,11 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
{
JsonExpr *jsexpr = (JsonExpr *) node;
- if (ExecEvalJsonNeedsSubTransaction(jsexpr, NULL))
+ /*
+ * XXX - don't really know why it makes sense to ignore the coercion
+ * part here.
+ */
+ if (ExecEvalJsonNeedsSubTransaction(jsexpr, false /* coerce */))
{
context->max_hazard = PROPARALLEL_UNSAFE;
return true;
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 358b831f7a..2639d7c73d 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -251,6 +251,7 @@ typedef enum ExprEvalOp
EEOP_JSONEXPR_COERCION,
EEOP_JSONEXPR_COERCION_ERROR,
EEOP_JSONEXPR_RESULT,
+ EEOP_JSON_ITEM_COERCION,
/* aggregation related nodes */
EEOP_AGG_STRICT_DESERIALIZE,
@@ -753,6 +754,13 @@ typedef struct ExprEvalStep
JsonExpr *jexpr;
int jump_skip_result;
} jsonexpr_result;
+
+ /* for EEOP_JSON_ITEM_COERCION */
+ struct
+ {
+ struct JsonItemCoercionsState *jcstate;
+ int jump_skip_item_coercion;
+ } json_item_coercion;
} d;
} ExprEvalStep;
@@ -833,10 +841,8 @@ typedef struct JsonExprPostEvalState
/* Cache for json_populate_type() called for coercion in some cases */
void *cache;
- /*
- * Coercion to be applied to JSON item returned by JsonPathValue(),
- * set by ExecPrepareJsonItemCoercion(). */
- struct JsonCoercionState *coercion;
+ /* Coercion state; same as parent JsonExprState.coercions when not NULL */
+ ExprState *coercions;
} JsonExprPostEvalState;
/*
@@ -875,26 +881,40 @@ typedef struct JsonExprState
Oid typioparam;
} input; /* I/O info for output type */
- struct JsonCoercionsState
- {
- struct JsonCoercionState
- {
- JsonCoercion *coercion; /* coercion expression */
- ExprState *estate; /* coercion expression state */
- } null,
- string,
- numeric,
- boolean,
- date,
- time,
- timetz,
- timestamp,
- timestamptz,
- composite;
- } coercions; /* states for coercion from SQL/JSON item
- * types directly to the output type */
+ /* To apply coercion to the final result if needed. */
+ ExprState *coercions;
} JsonExprState;
+/*
+ * State for a given member of JsonItemCoercions.
+ */
+typedef struct JsonItemCoercionState
+{
+ /* Expression used to evaluate the coercion */
+ JsonCoercion *coercion;
+
+ /* ExprEvalStep to compute this coercion's expression */
+ int jump_eval_expr;
+} JsonItemCoercionState;
+
+/*
+ * State for evaluating the coercion for a given JSON item using one of
+ * the following coercions.
+ */
+typedef struct JsonItemCoercionsState
+{
+ JsonItemCoercionState null;
+ JsonItemCoercionState string;
+ JsonItemCoercionState numeric;
+ JsonItemCoercionState boolean;
+ JsonItemCoercionState date;
+ JsonItemCoercionState time;
+ JsonItemCoercionState timetz;
+ JsonItemCoercionState timestamp;
+ JsonItemCoercionState timestamptz;
+ JsonItemCoercionState composite;
+} JsonItemCoercionsState;
+
/* functions in execExpr.c */
extern void ExprEvalPushStep(ExprState *es, const ExprEvalStep *s);
@@ -956,12 +976,11 @@ extern void ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
extern void ExecEvalJson(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
-extern Datum ExecPrepareJsonItemCoercion(struct JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pjcstate);
-extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *);
+JsonCoercion *ExecGetJsonItemCoercion(struct JsonbValue *item, JsonItemCoercions *coercions);
+extern Datum ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pjcstate);
+extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce);
extern Datum ExecEvalExprPassingCaseValue(ExprState *estate,
ExprContext *econtext, bool *isnull,
Datum caseval_datum,
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index bae466b523..6bdd9f5121 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -69,8 +69,10 @@ typedef enum
/* Convenience macros */
#define DatumGetJsonbP(d) ((Jsonb *) PG_DETOAST_DATUM(d))
+#define DatumGetJsonbValueP(d) ((JsonbValue *) DatumGetPointer(d))
#define DatumGetJsonbPCopy(d) ((Jsonb *) PG_DETOAST_DATUM_COPY(d))
#define JsonbPGetDatum(p) PointerGetDatum(p)
+#define JsonbValuePGetDatum(p) PointerGetDatum(p)
#define PG_GETARG_JSONB_P(x) DatumGetJsonbP(PG_GETARG_DATUM(x))
#define PG_GETARG_JSONB_P_COPY(x) DatumGetJsonbPCopy(PG_GETARG_DATUM(x))
#define PG_RETURN_JSONB_P(x) PG_RETURN_POINTER(x)
--
2.34.1
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-07-19 11:40 ` Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 75+ messages in thread
From: Amit Langote @ 2022-07-19 11:40 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Andres Freund <andres@anarazel.de>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On Tue, Jul 19, 2022 at 4:09 AM Andrew Dunstan <andrew@dunslane.net> wrote:
> On 2022-07-15 Fr 17:07, Andres Freund wrote:
> > Perhaps you could post your current state? I might be able to help resolving
> > some of the problems.
>
> Ok. Here is the state of things. This has proved to be rather more
> intractable than I expected. Almost all the legwork here has been done
> by Amit Langote, for which he deserves both my thanks and considerable
> credit, but I take responsibility for it.
>
> I just discovered today that this scheme is failing under
> "force_parallel_mode = regress". I have as yet no idea if that can be
> fixed simply or not.
The errors Andrew mentions here had to do with a bug of the new
coercion evaluation logic. The old code in ExecEvalJsonExpr() would
skip coercion evaluation and thus also the sub-transaction associated
with it for some JsonExprs that the new code would not and that didn't
sit well with the invariant that a parallel worker shouldn't try to
start a sub-transaction.
That bug has been fixed in the attached updated version.
> Apart from that I think the main outstanding issue
> is to fill in the gaps in llvm_compile_expr().
About that, I was wondering if the blocks in llvm_compile_expr() need
to be hand-coded to match what's added in ExecInterpExpr() or if I've
missed some tool that can be used instead?
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments: download all (3 files, .tar.gz)
v2-0002-Evaluate-various-JsonExpr-sub-expressions-using-p.patch (application/octet-stream, 35.8K) download
From bcaf5c0a90101205b698107d2bf87300be5b061d Mon Sep 17 00:00:00 2001
From: amitlan <amitlangote09@gmail.com>
Date: Tue, 5 Jul 2022 16:09:40 +0900
Subject: [PATCH v2 2/3] Evaluate various JsonExpr sub-expressions using parent
ExprState
These include PASSING args, ON ERROR, ON EMPTY default expressions,
and result_coercion expression.
To do so, this moves a bunch of code from ExecEvalJson(),
ExecEvalJsonExpr(), and ExecEvalJsonExprCoercion(), all of which
would previously run under the single step EEOP_JSONEXPR steps into
a number of new (sub-) ExprEvalSteps that are now added for a given
JsonExpr.
TODO: update llvm_compile_expr() to handle newly added
EEOP_JSONEXPR_* steps.
---
src/backend/executor/execExpr.c | 278 +++++++++++++++----
src/backend/executor/execExprInterp.c | 374 ++++++++++++++++++--------
src/backend/jit/llvm/llvmjit_expr.c | 35 ++-
src/include/executor/execExpr.h | 136 ++++++++--
4 files changed, 642 insertions(+), 181 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index a55e5000e2..4dbb24aaee 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2562,46 +2562,238 @@ ExecInitExprRec(Expr *node, ExprState *state,
{
JsonExpr *jexpr = castNode(JsonExpr, node);
JsonExprState *jsestate = palloc0(sizeof(JsonExprState));
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
ListCell *argexprlc;
ListCell *argnamelc;
+ int skip_step_off;
+ int onempty_step_off;
+ int onerror_step_off;
+ int coercion_step_off;
+ int coercion_error_step_off;
+ int result_step_off;
+ ExprEvalStep *as;
+ bool throwErrors =
+ (jexpr->on_error->btype == JSON_BEHAVIOR_ERROR);
- scratch.opcode = EEOP_JSONEXPR;
+ jsestate->jsexpr = jexpr;
+
+ /*
+ * Add steps to compute formatted_expr, pathspec, and
+ * PASSING arg expressions as things that must be
+ * evaluated before the "main" JSON path evaluation.
+ */
+ ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
+ &pre_eval->formatted_expr.value,
+ &pre_eval->formatted_expr.isnull);
+ ExecInitExprRec((Expr *) jexpr->path_spec, state,
+ &pre_eval->pathspec.value,
+ &pre_eval->pathspec.isnull);
+
+ /*
+ * However, before pushing steps for PASSING args, push a step
+ * that will decide whether to skip evaluating the args and
+ * JSON path evaluation depending on whether either of
+ * formatted_expr and pathspec is NULL.
+ */
+ scratch.opcode = EEOP_JSONEXPR_SKIP;
+ scratch.d.jsonexpr_skip.pre_eval = pre_eval;
+ scratch.d.jsonexpr_skip.post_eval = post_eval;
+ skip_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ jsestate->pre_eval.args = NIL;
+ forboth(argexprlc, jexpr->passing_values,
+ argnamelc, jexpr->passing_names)
+ {
+ Expr *argexpr = (Expr *) lfirst(argexprlc);
+ String *argname = lfirst_node(String, argnamelc);
+ JsonPathVariableEvalContext *var = palloc(sizeof(*var));
+
+ var->name = pstrdup(argname->sval);
+ var->typid = exprType((Node *) argexpr);
+ var->typmod = exprTypmod((Node *) argexpr);
+
+ /*
+ * Not necessary when being evaluated for a JsonExpr, like
+ * in this case.
+ */
+ var->estate = NULL;
+ var->econtext = NULL;
+ var->mcxt = NULL;
+
+ /*
+ * Mark these as always evaluated because they must have
+ * been evaluated before JSON path evaluation begins,
+ * because we haven't pushed the step for the latter yet.
+ */
+ var->evaluated = true;
+
+ ExecInitExprRec((Expr *) argexpr, state,
+ &var->value,
+ &var->isnull);
+
+ pre_eval->args = lappend(pre_eval->args, var);
+ }
+
+ /* Now push the step for the actual JSON path evaluation. */
+ scratch.opcode = EEOP_JSONEXPR_PATH;
scratch.d.jsonexpr.jsestate = jsestate;
+ ExprEvalPushStep(state, &scratch);
- jsestate->jsexpr = jexpr;
+ /*
+ * Now push steps to control the expressions evaluated based
+ * on the result of JSON path evaluation.
+ */
- jsestate->formatted_expr =
- palloc(sizeof(*jsestate->formatted_expr));
+ /*
+ * Step to handle ON EMPTY behavior correctly. Might be
+ * skipped over (along with default expression steps added
+ * below) if 1) the JSON path evaluation result is not empty,
+ * 2) ON ERROR behavior must be applied instead. Jump target
+ * addressesnecessary to do so will be set later.
+ */
+ onempty_step_off = -1;
+ if (jexpr->on_empty)
+ {
+ scratch.opcode = EEOP_JSONEXPR_ONEMPTY;
+ scratch.d.jsonexpr_onempty.post_eval = post_eval;
+ scratch.d.jsonexpr_onempty.jexpr = jexpr;
+ onempty_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
- ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
- &jsestate->formatted_expr->value,
- &jsestate->formatted_expr->isnull);
+ /* Push step(s) for the default ON EMPTY expression. */
+ if (jexpr->on_empty->default_expr)
+ ExecInitExprRec((Expr *) jexpr->on_empty->default_expr,
+ state, resv, resnull);
+ }
- jsestate->pathspec =
- palloc(sizeof(*jsestate->pathspec));
+ /*
+ * Step to handle ON ERROR behavior correctly. Might be
+ * skipped over (along with default expression steps added
+ * below) if no error occurs. Jump target address necessary
+ * to do so will be set later.
+ */
+ Assert(jexpr->on_error != NULL);
+ scratch.opcode = EEOP_JSONEXPR_ONERROR;
+ scratch.d.jsonexpr_onerror.post_eval = post_eval;
+ scratch.d.jsonexpr_onerror.jexpr = jexpr;
+ onerror_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
- ExecInitExprRec((Expr *) jexpr->path_spec, state,
- &jsestate->pathspec->value,
- &jsestate->pathspec->isnull);
+ /* Push step(s) for the default ON ERROR expression. */
+ if (jexpr->on_error->default_expr)
+ ExecInitExprRec((Expr *) jexpr->on_error->default_expr,
+ state, resv, resnull);
- jsestate->res_expr =
- palloc(sizeof(*jsestate->res_expr));
+ /*
+ * Step to handle applying coercion correctly. Might be
+ * skipped if no coercion needs to be applied separately
+ * from JSON path evaluation itself. Jump target address
+ * necessary to do so will be set later.
+ */
+ scratch.opcode = EEOP_JSONEXPR_COERCION;
+ scratch.d.jsonexpr_coercion.jsestate = jsestate;
+ scratch.d.jsonexpr_coercion.jump_skip_coercion = state->steps_len + 1;
+ coercion_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
- jsestate->result_expr = jexpr->result_coercion
- ? ExecInitExprWithCaseValue((Expr *) jexpr->result_coercion->expr,
- state->parent,
- &jsestate->res_expr->value,
- &jsestate->res_expr->isnull)
- : NULL;
+ /*
+ * Step to apply ON ERROR behavior if an error occurs when
+ * applying coercion. Will be skipped over (along with default
+ * expression steps added) if no error occurs. Jump target
+ * address necessary to do so will be set later.
+ */
+ scratch.opcode = EEOP_JSONEXPR_COERCION_ERROR;
+ scratch.d.jsonexpr_coercion_error.post_eval = post_eval;
+ scratch.d.jsonexpr_coercion_error.jexpr = jexpr;
+ coercion_error_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Push steps to evaluate the default ON ERROR expression. */
+ if (jexpr->on_error->default_expr)
+ ExecInitExprRec((Expr *) jexpr->on_error->default_expr,
+ state, resv, resnull);
+
+ /*
+ * Step to handle applying result_coercion if needed. Might
+ * be skipped if computing result_coercion is rendered
+ * unnecessary by earlier steps. Jump target address to do
+ * so will be set later.
+ */
+ result_step_off = state->steps_len;
+ if (jexpr->result_coercion && jexpr->result_coercion->expr)
+ {
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
+
+ scratch.opcode = EEOP_JSONEXPR_RESULT;
+ scratch.d.jsonexpr_result.post_eval = post_eval;
+ scratch.d.jsonexpr_result.jexpr = jexpr;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Steps to compute result_coercion expression. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
+
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) jexpr->result_coercion->expr,
+ state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
+ }
+
+ /*
+ * Adjust jump target addresses in various post-eval steps now
+ * that we have all the steps in place.
+ */
+
+ /* EEOP_JSONEXPR_SKIP */
+ as = &state->steps[skip_step_off];
+ as->d.jsonexpr_skip.jump_coercion = coercion_step_off;
+
+ /* EEOP_JSONEXPR_ONEMPTY */
+ if (onempty_step_off >= 0)
+ {
+ as = &state->steps[onempty_step_off];
+ as->d.jsonexpr_onempty.jump_error = onerror_step_off;
+ as->d.jsonexpr_onempty.jump_coercion = coercion_step_off;
+ }
+
+ /* EEOP_JSONEXPR_ONERROR */
+ as = &state->steps[onerror_step_off];
+ as->d.jsonexpr_onerror.jump_coercion = coercion_step_off;
- jsestate->default_on_empty = !jexpr->on_empty ? NULL :
- ExecInitExpr((Expr *) jexpr->on_empty->default_expr,
- state->parent);
+ /* EEOP_JSONEXPR_COERCION */
+ as = &state->steps[coercion_step_off];
+ as->d.jsonexpr_coercion.jump_coercion_error = coercion_error_step_off;
- jsestate->default_on_error =
- ExecInitExpr((Expr *) jexpr->on_error->default_expr,
- state->parent);
+ /* EEOP_JSONEXPR_COERCION_ERROR */
+ as = &state->steps[coercion_error_step_off];
+ as->d.jsonexpr_coercion_error.jump_coercion = coercion_step_off;
+ as->d.jsonexpr_coercion_error.jump_skip_coercion_error = result_step_off;
+ /* EEOP_JSONEXPR_RESULT */
+ as = &state->steps[result_step_off];
+ as->d.jsonexpr_result.jump_skip_result = state->steps_len;
+
+ /*
+ * Set if we must use a sub-transaction around path evaluation
+ * that can be aborted on error instead of aborting the parent
+ * query.
+ */
+ jsestate->needSubtrans =
+ ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ jsestate->coercionNeedSubtrans =
+ (!jsestate->needSubtrans && !throwErrors);
+
+ /*
+ * Set RETURNING type's input function used by
+ * ExecEvalJsonExprCoercion().
+ */
if (jexpr->omit_quotes ||
(jexpr->result_coercion && jexpr->result_coercion->via_io))
{
@@ -2616,31 +2808,7 @@ ExecInitExprRec(Expr *node, ExprState *state,
jsestate->input.finfo = finfo;
}
- jsestate->args = NIL;
-
- forboth(argexprlc, jexpr->passing_values,
- argnamelc, jexpr->passing_names)
- {
- Expr *argexpr = (Expr *) lfirst(argexprlc);
- String *argname = lfirst_node(String, argnamelc);
- JsonPathVariableEvalContext *var = palloc(sizeof(*var));
-
- var->name = pstrdup(argname->sval);
- var->typid = exprType((Node *) argexpr);
- var->typmod = exprTypmod((Node *) argexpr);
- var->estate = ExecInitExpr(argexpr, state->parent);
- var->econtext = NULL;
- var->mcxt = NULL;
- var->evaluated = false;
- var->value = (Datum) 0;
- var->isnull = true;
-
- jsestate->args =
- lappend(jsestate->args, var);
- }
-
- jsestate->cache = NULL;
-
+ /* Set up coercion related data structures. */
if (jexpr->coercions)
{
JsonCoercion **coercion;
@@ -2648,11 +2816,8 @@ ExecInitExprRec(Expr *node, ExprState *state,
Datum *caseval;
bool *casenull;
- jsestate->coercion_expr =
- palloc(sizeof(*jsestate->coercion_expr));
-
- caseval = &jsestate->coercion_expr->value;
- casenull = &jsestate->coercion_expr->isnull;
+ caseval = resv;
+ casenull = resnull;
for (cstate = &jsestate->coercions.null,
coercion = &jexpr->coercions->null;
@@ -2667,7 +2832,6 @@ ExecInitExprRec(Expr *node, ExprState *state,
}
}
- ExprEvalPushStep(state, &scratch);
break;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 0512a81c7c..5309017027 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -158,6 +158,15 @@ static TupleDesc get_cached_rowtype(Oid type_id, int32 typmod,
bool *changed);
static void ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, bool checkisnull);
+static Datum ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null);
+typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
+ Datum item, bool *resnull, void *p, bool *error);
+static Datum ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
+ ExprContext *econtext,
+ Datum res, bool *resnull,
+ void *p, bool *error, bool subtrans);
+static Datum ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
+ Datum res, bool *isNull, void *p, bool *error);
/* fast-path evaluation functions */
static Datum ExecJustInnerVar(ExprState *state, ExprContext *econtext, bool *isnull);
@@ -490,7 +499,13 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SUBPLAN,
&&CASE_EEOP_JSON_CONSTRUCTOR,
&&CASE_EEOP_IS_JSON,
- &&CASE_EEOP_JSONEXPR,
+ &&CASE_EEOP_JSONEXPR_SKIP,
+ &&CASE_EEOP_JSONEXPR_PATH,
+ &&CASE_EEOP_JSONEXPR_ONERROR,
+ &&CASE_EEOP_JSONEXPR_ONEMPTY,
+ &&CASE_EEOP_JSONEXPR_COERCION,
+ &&CASE_EEOP_JSONEXPR_COERCION_ERROR,
+ &&CASE_EEOP_JSONEXPR_RESULT,
&&CASE_EEOP_AGG_STRICT_DESERIALIZE,
&&CASE_EEOP_AGG_DESERIALIZE,
&&CASE_EEOP_AGG_STRICT_INPUT_CHECK_ARGS,
@@ -1817,13 +1832,182 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
- EEO_CASE(EEOP_JSONEXPR)
+ EEO_CASE(EEOP_JSONEXPR_SKIP)
+ {
+ JsonExprPreEvalState *pre_eval = op->d.jsonexpr_skip.pre_eval;
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_skip.post_eval;
+
+ /*
+ * Skip JSON evaluation if either of the input expressions has
+ * turned out to be NULL, though do execute domain checks for
+ * NULLs, which are handled by the coercion step.
+ */
+ if (pre_eval->formatted_expr.isnull || pre_eval->pathspec.isnull)
+ {
+ post_eval->coercion = NULL;
+ /* A signal to coercion step to not use a sub-trancaction. */
+ post_eval->coercion_error = true;
+ *op->resvalue = 0;
+ *op->resnull = true;
+ EEO_JUMP(op->d.jsonexpr_skip.jump_coercion);
+ }
+
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_PATH)
{
/* too complex for an inline implementation */
ExecEvalJson(state, op, econtext);
EEO_NEXT();
}
+ EEO_CASE(EEOP_JSONEXPR_ONEMPTY)
+ {
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_onempty.post_eval;
+ JsonExpr *jexpr = op->d.jsonexpr_onempty.jexpr;
+
+ /*
+ * Skip ON EMPTY handling if not empty or if it must be handled
+ * with a ON ERROR behavior instead.
+ */
+ if (post_eval->error)
+ EEO_JUMP(op->d.jsonexpr_onempty.jump_error);
+ else if (!post_eval->empty)
+ EEO_JUMP(op->d.jsonexpr_onempty.jump_coercion);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate
+ * value and skip over to the coercion step.
+ */
+ if (jexpr->on_empty->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ *op->resvalue = ExecEvalJsonBehavior(jexpr->on_empty,
+ op->resnull);
+ EEO_JUMP(op->d.jsonexpr_onempty.jump_coercion);
+ }
+
+ /*
+ * Evaluate the default ON EMPTY expression.
+ *
+ * Nothing for _COERCION and _RESULT steps to do as the
+ * expression we are about to compute has been coerced
+ * appropriately by the parser.
+ */
+ post_eval->coercion_done = true;
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_ONERROR)
+ {
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_onerror.post_eval;
+ JsonExpr *jexpr = op->d.jsonexpr_onerror.jexpr;
+
+ /* Nothing to do if no error has been found to begin with. */
+ if (!post_eval->error)
+ EEO_JUMP(op->d.jsonexpr_onerror.jump_coercion);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate
+ * value and skip over to the coercion step.
+ */
+ if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ /* Execute non-default ON ERROR behavior */
+ *op->resvalue = ExecEvalJsonBehavior(jexpr->on_error,
+ op->resnull);
+ EEO_JUMP(op->d.jsonexpr_onerror.jump_coercion);
+ }
+
+ /*
+ * Evaluate the default ON ERROR expression. See the note
+ * above regarding coercion.
+ */
+ post_eval->coercion_done = true;
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_COERCION_ERROR)
+ {
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_coercion_error.post_eval;
+ JsonExpr *jexpr = op->d.jsonexpr_coercion_error.jexpr;
+
+ /* Nothing to if no error was found when applying coercion. */
+ if (!post_eval->coercion_error)
+ EEO_JUMP(op->d.jsonexpr_coercion_error.jump_skip_coercion_error);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate
+ * value and skip over to the coercion step.
+ */
+ if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ /* Execute non-default ON ERROR behavior */
+ *op->resvalue = ExecEvalJsonBehavior(jexpr->on_error,
+ op->resnull);
+
+ /* Coercion the ON ERROR expression. */
+ post_eval->coercion_done = false;
+ post_eval->coercion = NULL;
+ EEO_JUMP(op->d.jsonexpr_coercion_error.jump_coercion);
+ }
+
+ /*
+ * Evaluate the default ON ERROR expression. See the note
+ * above regarding coercion.
+ */
+ post_eval->coercion_done = true;
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_COERCION)
+ {
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ bool coercionUseSubtrans;
+
+ /*
+ * Don't use a sub-transaction if coercing the ON ERROR expression
+ * after the previous coercion attempt caused an error or if got
+ * here from EEOP_JSONEXPR_SKIP, so that any that any errors that
+ * are encountered downstream this time are thrown there, that is,
+ * not rethrown to be handled here.
+ *
+ * XXX - doesn't that violate ON ERROR behavior? Actually, that is
+ * how it behaved even before this expression eval logic rewrite!
+ */
+ coercionUseSubtrans =
+ (jsestate->coercionNeedSubtrans &&
+ !post_eval->coercion_error);
+
+ /* ExecEvalJson() may have found coercion to be unnecessary. */
+ if (post_eval->coercion_done)
+ EEO_JUMP(op->d.jsonexpr_coercion.jump_skip_coercion);
+
+ post_eval->coercion_error = false;
+ *op->resvalue =
+ ExecEvalJsonExprSubtrans(ExecEvalJsonExprCoercion, op, econtext,
+ *op->resvalue,
+ op->resnull, NULL,
+ &post_eval->coercion_error,
+ coercionUseSubtrans);
+ if (post_eval->coercion_error)
+ EEO_JUMP(op->d.jsonexpr_coercion.jump_coercion_error);
+
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_RESULT)
+ {
+ JsonExprPostEvalState *post_eval = op->d.jsonexpr_result.post_eval;
+
+ if (post_eval->coercion_done)
+ EEO_JUMP(op->d.jsonexpr_result.jump_skip_result);
+
+ /* Evaluate result_coercion expression. */
+ EEO_NEXT();
+ }
+
EEO_CASE(EEOP_LAST)
{
/* unreachable */
@@ -4602,8 +4786,7 @@ ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
* Evaluate a JSON error/empty behavior result.
*/
static Datum
-ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
- ExprState *default_estate, bool *is_null)
+ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null)
{
*is_null = false;
@@ -4628,7 +4811,9 @@ ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
return (Datum) 0;
case JSON_BEHAVIOR_DEFAULT:
- return ExecEvalExpr(default_estate, econtext, is_null);
+ /* Always handled in the caller. */
+ Assert(false);
+ return (Datum) 0;
default:
elog(ERROR, "unrecognized SQL/JSON behavior %d", behavior->btype);
@@ -4643,18 +4828,25 @@ static Datum
ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
Datum res, bool *isNull, void *p, bool *error)
{
- ExprState *estate = p;
- JsonExprState *jsestate;
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ JsonExpr *jexpr = jsestate->jsexpr;
+ ExprState *estate = post_eval->coercion ?
+ post_eval->coercion->estate : NULL;
+
+ /*
+ * Initially assume we will indeed perform the coercion either
+ * using post_eval->coercion, with the input function for the
+ * output type, or with json_populate_type().
+ */
+ post_eval->coercion_done = true;
if (estate) /* coerce using specified expression */
return ExecEvalExpr(estate, econtext, isNull);
- jsestate = op->d.jsonexpr.jsestate;
-
- if (jsestate->jsexpr->op != JSON_EXISTS_OP)
+ if (jexpr->op != JSON_EXISTS_OP)
{
- JsonCoercion *coercion = jsestate->jsexpr->result_coercion;
- JsonExpr *jexpr = jsestate->jsexpr;
+ JsonCoercion *coercion = jexpr->result_coercion;
Jsonb *jb = *isNull ? NULL : DatumGetJsonbP(res);
if ((coercion && coercion->via_io) ||
@@ -4672,18 +4864,16 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
return json_populate_type(res, JSONBOID,
jexpr->returning->typid,
jexpr->returning->typmod,
- &jsestate->cache,
+ &post_eval->cache,
econtext->ecxt_per_query_memory,
isNull);
}
- if (jsestate->result_expr)
- {
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *isNull;
-
- res = ExecEvalExpr(jsestate->result_expr, econtext, isNull);
- }
+ /*
+ * Let the caller know that no coercion was done here, so it can
+ * coerce with jexpr->result_coercion if there's one.
+ */
+ post_eval->coercion_done = false;
return res;
}
@@ -4717,17 +4907,30 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
if (!var)
return -1;
- if (!var->evaluated)
+ /*
+ * When belonging to a JsonExpr, path variables are computed with the
+ * JsonExpr's ExprState (var->estate is NULL), so don't need to be computed
+ * here. In some other cases, such as when the path variables belonging
+ * to a JsonTable instead, those variables must be evaluated on their own,
+ * without the enclosing JsonExpr itself needing to be evaluated, so must
+ * be handled here.
+ */
+ if (var->estate && !var->evaluated)
{
MemoryContext oldcxt = var->mcxt ?
MemoryContextSwitchTo(var->mcxt) : NULL;
+ Assert(var->econtext != NULL);
var->value = ExecEvalExpr(var->estate, var->econtext, &var->isnull);
var->evaluated = true;
if (oldcxt)
MemoryContextSwitchTo(oldcxt);
}
+ else
+ {
+ Assert(var->evaluated);
+ }
if (var->isnull)
{
@@ -4832,9 +5035,6 @@ ExecPrepareJsonItemCoercion(JsonbValue *item,
return res;
}
-typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
- Datum item, bool *resnull, void *p, bool *error);
-
static Datum
ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
ExprContext *econtext,
@@ -4905,7 +5105,6 @@ typedef struct
{
JsonPath *path;
bool *error;
- bool coercionInSubtrans;
} ExecEvalJsonExprContext;
static Datum
@@ -4916,16 +5115,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
ExecEvalJsonExprContext *cxt = pcxt;
JsonPath *path = cxt->path;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
- ExprState *estate = NULL;
- bool empty = false;
+ bool *empty = &post_eval->empty;
Datum res = (Datum) 0;
switch (jexpr->op)
{
case JSON_QUERY_OP:
- res = JsonPathQuery(item, path, jexpr->wrapper, &empty, error,
- jsestate->args);
+ res = JsonPathQuery(item, path, jexpr->wrapper, empty, error,
+ pre_eval->args);
if (error && *error)
{
*resnull = true;
@@ -4937,16 +5137,19 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
case JSON_VALUE_OP:
{
struct JsonCoercionState *jcstate;
- JsonbValue *jbv = JsonPathValue(item, path, &empty, error,
- jsestate->args);
+ JsonbValue *jbv = JsonPathValue(item, path, empty, error,
+ pre_eval->args);
if (error && *error)
+ {
+ *resnull = true;
return (Datum) 0;
+ }
if (!jbv) /* NULL or empty */
break;
- Assert(!empty);
+ Assert(!*empty);
*resnull = false;
@@ -4964,7 +5167,6 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
jsestate->jsexpr->returning,
&jsestate->coercions,
&jcstate);
-
if (jcstate->coercion &&
(jcstate->coercion->via_io ||
jcstate->coercion->via_populate))
@@ -4972,6 +5174,7 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
if (error)
{
*error = true;
+ *resnull = true;
return (Datum) 0;
}
@@ -4983,32 +5186,34 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_SQL_JSON_ITEM_CANNOT_BE_CAST_TO_TARGET_TYPE),
errmsg("SQL/JSON item cannot be cast to target type")));
}
- else if (!jcstate->estate)
- return res; /* no coercion */
-
- /* coerce using specific expression */
- estate = jcstate->estate;
- jsestate->coercion_expr->value = res;
- jsestate->coercion_expr->isnull = *resnull;
+ else if (jcstate->estate == NULL)
+ {
+ /* No coercion needed */
+ post_eval->coercion_done = true;
+ }
+ else
+ {
+ /* Coerce using specific expression */
+ post_eval->coercion = jcstate;
+ }
break;
}
case JSON_EXISTS_OP:
{
bool exists = JsonPathExists(item, path,
- jsestate->args,
+ pre_eval->args,
error);
*resnull = error && *error;
res = BoolGetDatum(exists);
- if (!jsestate->result_expr)
+ if (jexpr->result_coercion == NULL)
+ {
+ /* No coercion needed */
+ post_eval->coercion_done = true;
return res;
-
- /* coerce using result expression */
- estate = jsestate->result_expr;
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *resnull;
+ }
break;
}
@@ -5021,7 +5226,11 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
return (Datum) 0;
}
- if (empty)
+ /*
+ * If the ON EMPTY behavior is to cause an error, do so here. Other
+ * behaviors will be handled by the caller.
+ */
+ if (*empty)
{
Assert(jexpr->on_empty); /* it is not JSON_EXISTS */
@@ -5037,24 +5246,9 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_NO_SQL_JSON_ITEM),
errmsg("no SQL/JSON item")));
}
-
- if (jexpr->on_empty->btype == JSON_BEHAVIOR_DEFAULT)
-
- /*
- * Execute DEFAULT expression as a coercion expression, because
- * its result is already coerced to the target type.
- */
- estate = jsestate->default_on_empty;
- else
- /* Execute ON EMPTY behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_empty,
- jsestate->default_on_empty,
- resnull);
}
- return ExecEvalJsonExprSubtrans(ExecEvalJsonExprCoercion, op, econtext,
- res, resnull, estate, error,
- cxt->coercionInSubtrans);
+ return res;
}
bool
@@ -5082,64 +5276,30 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
{
ExecEvalJsonExprContext cxt;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
Datum item;
Datum res = (Datum) 0;
JsonPath *path;
- ListCell *lc;
- bool error = false;
- bool needSubtrans;
bool throwErrors = jexpr->on_error->btype == JSON_BEHAVIOR_ERROR;
*op->resnull = true; /* until we get a result */
*op->resvalue = (Datum) 0;
- if (jsestate->formatted_expr->isnull || jsestate->pathspec->isnull)
- {
- /* execute domain checks for NULLs */
- (void) ExecEvalJsonExprCoercion(op, econtext, res, op->resnull,
- NULL, NULL);
-
- Assert(*op->resnull);
- return;
- }
+ item = pre_eval->formatted_expr.value;
+ path = DatumGetJsonPathP(pre_eval->pathspec.value);
- item = jsestate->formatted_expr->value;
- path = DatumGetJsonPathP(jsestate->pathspec->value);
-
- /* reset JSON path variable contexts */
- foreach(lc, jsestate->args)
- {
- JsonPathVariableEvalContext *var = lfirst(lc);
-
- var->econtext = econtext;
- var->evaluated = false;
- }
-
- needSubtrans = ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ /* Reset JsonExprPostEvalState for this evaluation. */
+ memset(post_eval, 0, sizeof(*post_eval));
cxt.path = path;
- cxt.error = throwErrors ? NULL : &error;
- cxt.coercionInSubtrans = !needSubtrans && !throwErrors;
- Assert(!needSubtrans || cxt.error);
+ cxt.error = throwErrors ? NULL : &post_eval->error;
+ Assert(!jsestate->needSubtrans || cxt.error);
res = ExecEvalJsonExprSubtrans(ExecEvalJsonExpr, op, econtext, item,
op->resnull, &cxt, cxt.error,
- needSubtrans);
-
- if (error)
- {
- /* Execute ON ERROR behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_error,
- jsestate->default_on_error,
- op->resnull);
-
- /* result is already coerced in DEFAULT behavior case */
- if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
- res = ExecEvalJsonExprCoercion(op, econtext, res,
- op->resnull,
- NULL, NULL);
- }
+ jsestate->needSubtrans);
*op->resvalue = res;
}
diff --git a/src/backend/jit/llvm/llvmjit_expr.c b/src/backend/jit/llvm/llvmjit_expr.c
index b6b6512ef1..c133600005 100644
--- a/src/backend/jit/llvm/llvmjit_expr.c
+++ b/src/backend/jit/llvm/llvmjit_expr.c
@@ -2359,11 +2359,44 @@ llvm_compile_expr(ExprState *state)
LLVMBuildBr(b, opblocks[opno + 1]);
break;
- case EEOP_JSONEXPR:
+ case EEOP_JSONEXPR_SKIP:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_PATH:
build_EvalXFunc(b, mod, "ExecEvalJson",
v_state, op, v_econtext);
LLVMBuildBr(b, opblocks[opno + 1]);
break;
+ case EEOP_JSONEXPR_ONERROR:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_ONEMPTY:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_COERCION:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_COERCION_ERROR:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_RESULT:
+ {
+ /*
+ * Perform the step at jsonexpr_result.jumpnext if the
+ * coercion already done, which effectively skips the
+ * step(s) to perform the coercion using the result
+ * expression.
+ * XXX - Hoping this DTRT.
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b, LLVMIntEQ,
+ l_sbool_const(op->d.jsonexpr_result.post_eval->coercion_done),
+ l_sbool_const(1),
+ ""),
+ opblocks[op->d.jsonexpr_result.jump_skip_result],
+ opblocks[opno + 1]);
+ }
+ break;
case EEOP_LAST:
Assert(false);
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index e55a572854..358b831f7a 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -244,7 +244,13 @@ typedef enum ExprEvalOp
EEOP_SUBPLAN,
EEOP_JSON_CONSTRUCTOR,
EEOP_IS_JSON,
- EEOP_JSONEXPR,
+ EEOP_JSONEXPR_SKIP,
+ EEOP_JSONEXPR_PATH,
+ EEOP_JSONEXPR_ONERROR,
+ EEOP_JSONEXPR_ONEMPTY,
+ EEOP_JSONEXPR_COERCION,
+ EEOP_JSONEXPR_COERCION_ERROR,
+ EEOP_JSONEXPR_RESULT,
/* aggregation related nodes */
EEOP_AGG_STRICT_DESERIALIZE,
@@ -682,12 +688,71 @@ typedef struct ExprEvalStep
JsonIsPredicate *pred; /* original expression node */
} is_json;
- /* for EEOP_JSONEXPR */
+ /* for EEOP_JSONEXPR_PATH */
struct
{
struct JsonExprState *jsestate;
} jsonexpr;
+ /* for EEOP_JSONEXPR_SKIP */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprPreEvalState *pre_eval;
+ struct JsonExprPostEvalState *post_eval;
+ int jump_coercion;
+ } jsonexpr_skip;
+
+ /* for EEOP_JSONEXPR_ONEMPTY */
+ struct
+ {
+ /* Points to jsonexpr.jsestate->post_eval */
+ struct JsonExprPostEvalState *post_eval;
+ /* Same as jsonexpr.jsestate->jsexpr */
+ JsonExpr *jexpr;
+ int jump_error;
+ int jump_coercion;
+ } jsonexpr_onempty;
+
+ /* for EEOP_JSONEXPR_ONERROR */
+ struct
+ {
+ /* Points to jsonexpr.jsestate->post_eval */
+ struct JsonExprPostEvalState *post_eval;
+ /* Same as jsonexpr.jsestate->jsexpr */
+ JsonExpr *jexpr;
+ int jump_coercion;
+ } jsonexpr_onerror;
+
+ /* for EEOP_JSONEXPR_COERCION */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_skip_coercion;
+ int jump_coercion_error;
+ } jsonexpr_coercion;
+
+ /* for EEOP_JSONEXPR_COERCION_ERROR */
+ struct
+ {
+ /* Points to jsonexpr.jsestate->post_eval */
+ struct JsonExprPostEvalState *post_eval;
+ /* Same as jsonexpr.jsestate->jsexpr */
+ JsonExpr *jexpr;
+ int jump_skip_coercion_error;
+ int jump_coercion;
+ } jsonexpr_coercion_error;
+
+ /* for EEOP_JSONEXPR_RESULT */
+ struct
+ {
+ /* Points to jsonexpr.jsestate->post_eval */
+ struct JsonExprPostEvalState *post_eval;
+ /* Same as jsonexpr.jsestate->jsexpr */
+ JsonExpr *jexpr;
+ int jump_skip_result;
+ } jsonexpr_result;
} d;
} ExprEvalStep;
@@ -747,30 +812,69 @@ typedef struct JsonConstructorExprState
int nargs;
} JsonConstructorExprState;
+/*
+ * State for (EEOP_JSONEXPR_) ONERROR, ONEMPTY, COERCION, COERCION_ERROR,
+ * and RESULT steps that will run after EEOP_JSONEXPR_PATH.
+ */
+typedef struct JsonExprPostEvalState
+{
+ /* Is JSON item empty? */
+ bool empty;
+
+ /* Did JSON item evaluation cause an error? */
+ bool error;
+
+ /* Did coercion evaluation cause an error? */
+ bool coercion_error;
+
+ /* Has the result been coerced properly? */
+ bool coercion_done;
+
+ /* Cache for json_populate_type() called for coercion in some cases */
+ void *cache;
+
+ /*
+ * Coercion to be applied to JSON item returned by JsonPathValue(),
+ * set by ExecPrepareJsonItemCoercion(). */
+ struct JsonCoercionState *coercion;
+} JsonExprPostEvalState;
+
+/*
+ * Information computed before evaluating a JsonExpr expression.
+ */
+typedef struct JsonExprPreEvalState
+{
+ /* value/isnull for JsonExpr.formatted_expr */
+ NullableDatum formatted_expr;
+
+ /* value/isnull for JsonExpr.pathspec */
+ NullableDatum pathspec;
+
+ /* JsonPathVariableEvalContext entries for JsonExpr.passing_values */
+ List *args;
+} JsonExprPreEvalState;
+
/* EEOP_JSONEXPR state, too big to inline */
typedef struct JsonExprState
{
JsonExpr *jsexpr; /* original expression node */
+ JsonExprPreEvalState pre_eval;
+ JsonExprPostEvalState post_eval;
+
+ /*
+ * Should use a sub-transaction for path evaluation and subsequent
+ * coercion evaluation, if any?
+ */
+ bool needSubtrans;
+ bool coercionNeedSubtrans;
+
struct
{
FmgrInfo *finfo; /* typinput function for output type */
Oid typioparam;
} input; /* I/O info for output type */
- NullableDatum
- *formatted_expr, /* formatted context item value */
- *res_expr, /* result item */
- *coercion_expr, /* input for JSON item coercion */
- *pathspec; /* path specification value */
-
- ExprState *result_expr; /* coerced to output type */
- ExprState *default_on_empty; /* ON EMPTY DEFAULT expression */
- ExprState *default_on_error; /* ON ERROR DEFAULT expression */
- List *args; /* passing arguments */
-
- void *cache; /* cache for json_populate_type() */
-
struct JsonCoercionsState
{
struct JsonCoercionState
@@ -779,7 +883,7 @@ typedef struct JsonExprState
ExprState *estate; /* coercion expression state */
} null,
string,
- numeric ,
+ numeric,
boolean,
date,
time,
--
2.35.3
v2-0003-Use-one-ExprState-to-implement-JsonItemCoercions.patch (application/octet-stream, 17.1K) download
From fd2b9dbab24622e741108ca55a3f1fcae3c1719b Mon Sep 17 00:00:00 2001
From: amitlan <amitlangote09@gmail.com>
Date: Fri, 15 Jul 2022 12:57:50 +0900
Subject: [PATCH v2 3/3] Use one ExprState to implement JsonItemCoercions
Current implementation assigns one ExprState for every JsonCoercion
member of JsonItemCoercions, which seems wasteful.
This commit instead assigns the ExprState to JsonItemCoercions itself
and arranges to compute the member JsonCoercion's expressions using
ExprEvalSteps instead.
---
src/backend/executor/execExpr.c | 113 ++++++++++++++++++----
src/backend/executor/execExprInterp.c | 133 ++++++++++++++++++++------
src/backend/jit/llvm/llvmjit_expr.c | 3 +
src/backend/optimizer/util/clauses.c | 6 +-
src/include/executor/execExpr.h | 75 +++++++++------
src/include/utils/jsonb.h | 2 +
6 files changed, 256 insertions(+), 76 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 4dbb24aaee..be98544e05 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2786,7 +2786,8 @@ ExecInitExprRec(Expr *node, ExprState *state,
* query.
*/
jsestate->needSubtrans =
- ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ ExecEvalJsonNeedsSubTransaction(jexpr,
+ true /* coerce */);
jsestate->coercionNeedSubtrans =
(!jsestate->needSubtrans && !throwErrors);
@@ -2810,26 +2811,102 @@ ExecInitExprRec(Expr *node, ExprState *state,
/* Set up coercion related data structures. */
if (jexpr->coercions)
+ jsestate->coercions =
+ ExecInitExprWithCaseValue((Expr *) jexpr->coercions,
+ state->parent,
+ resv, resnull);
+
+ break;
+ }
+
+ case T_JsonItemCoercions:
+ {
+ JsonItemCoercions *coercions = castNode(JsonItemCoercions,
+ node);
+ JsonItemCoercionsState *jcstate =
+ palloc0(sizeof(JsonItemCoercionsState));
+ JsonCoercion **coercion;
+ JsonItemCoercionState *cstate;
+ ExprEvalStep *as;
+ int json_item_coercion_step_id;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
+
+ /*
+ * First push a step to read the value provided by the parent
+ * JsonExpr via a CaseTestExpr.
+ */
+ scratch.opcode = EEOP_CASE_TESTVAL;
+ scratch.d.casetest.value = state->innermost_caseval;
+ scratch.d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Push the control step. */
+ scratch.opcode = EEOP_JSON_ITEM_COERCION;
+ scratch.d.json_item_coercion.jcstate = jcstate;
+ json_item_coercion_step_id = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+ /* Will set jump_skip_coercion target address below. */
+
+ /*
+ * Now push the steps of individual coercion's expression, if
+ * needed.
+ */
+ for (cstate = &jcstate->null,
+ coercion = &coercions->null;
+ coercion <= &coercions->composite;
+ coercion++, cstate++)
{
- JsonCoercion **coercion;
- struct JsonCoercionState *cstate;
- Datum *caseval;
- bool *casenull;
-
- caseval = resv;
- casenull = resnull;
-
- for (cstate = &jsestate->coercions.null,
- coercion = &jexpr->coercions->null;
- coercion <= &jexpr->coercions->composite;
- coercion++, cstate++)
+ cstate->coercion = *coercion;
+ if (cstate->coercion && cstate->coercion->expr)
{
- cstate->coercion = *coercion;
- cstate->estate = *coercion ?
- ExecInitExprWithCaseValue((Expr *) (*coercion)->expr,
- state->parent,
- caseval, casenull) : NULL;
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
+
+ cstate->jump_eval_expr = state->steps_len;
+
+ /* Push step(s) to compute (*coercion)->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
+
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) cstate->coercion->expr,
+ state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
}
+ else
+ cstate->jump_eval_expr = -1;
+
+ /* Emit JUMP step to jump to end of coercions code */
+ scratch.opcode = EEOP_JUMP;
+
+ /*
+ * Remember JUMP step address to set the actual jump
+ * target address below.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len);
+ ExprEvalPushStep(state, &scratch);
+ }
+
+ /*
+ * Adjust the jump target address of the control step and all
+ * the jumps we added in the above loop to make them point to
+ * the step after the last step that would have been added
+ * above.
+ */
+ as = &state->steps[json_item_coercion_step_id];
+ as->d.json_item_coercion.jump_skip_item_coercion = state->steps_len;
+ foreach(lc, adjust_jumps)
+ {
+ int jump_step_id = lfirst_int(lc);
+
+ as = &state->steps[jump_step_id];
+ as->d.jump.jumpdone = state->steps_len;
}
break;
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 5309017027..144536465a 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -506,6 +506,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_JSONEXPR_COERCION,
&&CASE_EEOP_JSONEXPR_COERCION_ERROR,
&&CASE_EEOP_JSONEXPR_RESULT,
+ &&CASE_EEOP_JSON_ITEM_COERCION,
&&CASE_EEOP_AGG_STRICT_DESERIALIZE,
&&CASE_EEOP_AGG_DESERIALIZE,
&&CASE_EEOP_AGG_STRICT_INPUT_CHECK_ARGS,
@@ -1844,7 +1845,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
*/
if (pre_eval->formatted_expr.isnull || pre_eval->pathspec.isnull)
{
- post_eval->coercion = NULL;
+ post_eval->coercions = NULL;
/* A signal to coercion step to not use a sub-trancaction. */
post_eval->coercion_error = true;
*op->resvalue = 0;
@@ -1948,7 +1949,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
/* Coercion the ON ERROR expression. */
post_eval->coercion_done = false;
- post_eval->coercion = NULL;
+ post_eval->coercions = NULL;
EEO_JUMP(op->d.jsonexpr_coercion_error.jump_coercion);
}
@@ -2008,6 +2009,23 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
+ EEO_CASE(EEOP_JSON_ITEM_COERCION)
+ {
+ JsonItemCoercionsState *jcstate = op->d.json_item_coercion.jcstate;
+ JsonItemCoercionState *cstate = NULL;
+
+ *op->resvalue =
+ ExecPrepareJsonItemCoercion(*op->resvalue, jcstate, &cstate);
+ if (cstate->coercion && cstate->coercion->expr)
+ {
+ Assert(cstate->jump_eval_expr >= 0);
+ EEO_JUMP(cstate->jump_eval_expr);
+ }
+
+ /* Skip over all of the steps added for this JsonItemCoercions. */
+ EEO_JUMP(op->d.json_item_coercion.jump_skip_item_coercion);
+ }
+
EEO_CASE(EEOP_LAST)
{
/* unreachable */
@@ -4831,8 +4849,7 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
- ExprState *estate = post_eval->coercion ?
- post_eval->coercion->estate : NULL;
+ ExprState *estate = post_eval->coercions;
/*
* Initially assume we will indeed perform the coercion either
@@ -4944,19 +4961,84 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
return id;
}
+/*
+ * Return a coercion among those given in 'coercions' for given
+ * JSON item.
+ */
+JsonCoercion *
+ExecGetJsonItemCoercion(JsonbValue *item, JsonItemCoercions *coercions)
+{
+ if (item->type == jbvBinary &&
+ JsonContainerIsScalar(item->val.binary.data))
+ {
+ JsonbValue buf;
+ bool res PG_USED_FOR_ASSERTS_ONLY;
+
+ res = JsonbExtractScalar(item->val.binary.data, &buf);
+ item = &buf;
+ Assert(res);
+ }
+
+ /* get coercion state reference and datum of the corresponding SQL type */
+ switch (item->type)
+ {
+ case jbvNull:
+ return coercions->null;
+
+ case jbvString:
+ return coercions->string;
+
+ case jbvNumeric:
+ return coercions->numeric;
+
+ case jbvBool:
+ return coercions->boolean;
+
+ case jbvDatetime:
+ switch (item->val.datetime.typid)
+ {
+ case DATEOID:
+ return coercions->date;
+ case TIMEOID:
+ return coercions->time;
+ case TIMETZOID:
+ return coercions->timetz;
+ case TIMESTAMPOID:
+ return coercions->timestamp;
+ case TIMESTAMPTZOID:
+ return coercions->timestamptz;
+ default:
+ elog(ERROR, "unexpected jsonb datetime type oid %u",
+ item->val.datetime.typid);
+ }
+ break;
+
+ case jbvArray:
+ case jbvObject:
+ case jbvBinary:
+ return coercions->composite;
+
+ default:
+ elog(ERROR, "unexpected jsonb value type %d", item->type);
+ }
+
+ Assert(false);
+ return NULL;
+}
+
/*
* Prepare SQL/JSON item coercion to the output type. Returned a datum of the
* corresponding SQL type and a pointer to the coercion state.
*/
Datum
-ExecPrepareJsonItemCoercion(JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pcoercion)
+ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pcoercion)
{
- struct JsonCoercionState *coercion;
+ JsonbValue *item = DatumGetJsonbValueP(itemval);
+ JsonItemCoercionState *coercion;
Datum res;
- JsonbValue buf;
+ JsonbValue buf;
if (item->type == jbvBinary &&
JsonContainerIsScalar(item->val.binary.data))
@@ -5136,7 +5218,7 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
case JSON_VALUE_OP:
{
- struct JsonCoercionState *jcstate;
+ JsonCoercion *coercion;
JsonbValue *jbv = JsonPathValue(item, path, empty, error,
pre_eval->args);
@@ -5162,14 +5244,12 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
break;
}
- /* Use coercion from SQL/JSON item type to the output type */
- res = ExecPrepareJsonItemCoercion(jbv,
- jsestate->jsexpr->returning,
- &jsestate->coercions,
- &jcstate);
- if (jcstate->coercion &&
- (jcstate->coercion->via_io ||
- jcstate->coercion->via_populate))
+ /*
+ * Error out no cast exists to coerce SQL/JSON item to the
+ * the output type
+ */
+ coercion = ExecGetJsonItemCoercion(jbv, jsestate->jsexpr->coercions);
+ if (coercion && (coercion->via_io || coercion->via_populate))
{
if (error)
{
@@ -5186,15 +5266,11 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_SQL_JSON_ITEM_CANNOT_BE_CAST_TO_TARGET_TYPE),
errmsg("SQL/JSON item cannot be cast to target type")));
}
- else if (jcstate->estate == NULL)
- {
- /* No coercion needed */
- post_eval->coercion_done = true;
- }
else
{
- /* Coerce using specific expression */
- post_eval->coercion = jcstate;
+ /* Coerce using specific expression. */
+ res = JsonbValuePGetDatum(jbv);
+ post_eval->coercions = jsestate->coercions;
}
break;
}
@@ -5252,8 +5328,7 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
}
bool
-ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *coercions)
+ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce)
{
if (jsexpr->on_error->btype == JSON_BEHAVIOR_ERROR)
return false;
@@ -5261,7 +5336,7 @@ ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
if (jsexpr->op == JSON_EXISTS_OP && !jsexpr->result_coercion)
return false;
- if (!coercions)
+ if (!coerce)
return true;
return false;
diff --git a/src/backend/jit/llvm/llvmjit_expr.c b/src/backend/jit/llvm/llvmjit_expr.c
index c133600005..5f06a87373 100644
--- a/src/backend/jit/llvm/llvmjit_expr.c
+++ b/src/backend/jit/llvm/llvmjit_expr.c
@@ -2397,6 +2397,9 @@ llvm_compile_expr(ExprState *state)
opblocks[opno + 1]);
}
break;
+ case EEOP_JSON_ITEM_COERCION:
+ /* XXX - handle! */
+ break;
case EEOP_LAST:
Assert(false);
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..9c02218155 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -901,7 +901,11 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
{
JsonExpr *jsexpr = (JsonExpr *) node;
- if (ExecEvalJsonNeedsSubTransaction(jsexpr, NULL))
+ /*
+ * XXX - don't really know why it makes sense to ignore the coercion
+ * part here.
+ */
+ if (ExecEvalJsonNeedsSubTransaction(jsexpr, false /* coerce */))
{
context->max_hazard = PROPARALLEL_UNSAFE;
return true;
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 358b831f7a..2639d7c73d 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -251,6 +251,7 @@ typedef enum ExprEvalOp
EEOP_JSONEXPR_COERCION,
EEOP_JSONEXPR_COERCION_ERROR,
EEOP_JSONEXPR_RESULT,
+ EEOP_JSON_ITEM_COERCION,
/* aggregation related nodes */
EEOP_AGG_STRICT_DESERIALIZE,
@@ -753,6 +754,13 @@ typedef struct ExprEvalStep
JsonExpr *jexpr;
int jump_skip_result;
} jsonexpr_result;
+
+ /* for EEOP_JSON_ITEM_COERCION */
+ struct
+ {
+ struct JsonItemCoercionsState *jcstate;
+ int jump_skip_item_coercion;
+ } json_item_coercion;
} d;
} ExprEvalStep;
@@ -833,10 +841,8 @@ typedef struct JsonExprPostEvalState
/* Cache for json_populate_type() called for coercion in some cases */
void *cache;
- /*
- * Coercion to be applied to JSON item returned by JsonPathValue(),
- * set by ExecPrepareJsonItemCoercion(). */
- struct JsonCoercionState *coercion;
+ /* Coercion state; same as parent JsonExprState.coercions when not NULL */
+ ExprState *coercions;
} JsonExprPostEvalState;
/*
@@ -875,26 +881,40 @@ typedef struct JsonExprState
Oid typioparam;
} input; /* I/O info for output type */
- struct JsonCoercionsState
- {
- struct JsonCoercionState
- {
- JsonCoercion *coercion; /* coercion expression */
- ExprState *estate; /* coercion expression state */
- } null,
- string,
- numeric,
- boolean,
- date,
- time,
- timetz,
- timestamp,
- timestamptz,
- composite;
- } coercions; /* states for coercion from SQL/JSON item
- * types directly to the output type */
+ /* To apply coercion to the final result if needed. */
+ ExprState *coercions;
} JsonExprState;
+/*
+ * State for a given member of JsonItemCoercions.
+ */
+typedef struct JsonItemCoercionState
+{
+ /* Expression used to evaluate the coercion */
+ JsonCoercion *coercion;
+
+ /* ExprEvalStep to compute this coercion's expression */
+ int jump_eval_expr;
+} JsonItemCoercionState;
+
+/*
+ * State for evaluating the coercion for a given JSON item using one of
+ * the following coercions.
+ */
+typedef struct JsonItemCoercionsState
+{
+ JsonItemCoercionState null;
+ JsonItemCoercionState string;
+ JsonItemCoercionState numeric;
+ JsonItemCoercionState boolean;
+ JsonItemCoercionState date;
+ JsonItemCoercionState time;
+ JsonItemCoercionState timetz;
+ JsonItemCoercionState timestamp;
+ JsonItemCoercionState timestamptz;
+ JsonItemCoercionState composite;
+} JsonItemCoercionsState;
+
/* functions in execExpr.c */
extern void ExprEvalPushStep(ExprState *es, const ExprEvalStep *s);
@@ -956,12 +976,11 @@ extern void ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
extern void ExecEvalJson(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
-extern Datum ExecPrepareJsonItemCoercion(struct JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pjcstate);
-extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *);
+JsonCoercion *ExecGetJsonItemCoercion(struct JsonbValue *item, JsonItemCoercions *coercions);
+extern Datum ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pjcstate);
+extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce);
extern Datum ExecEvalExprPassingCaseValue(ExprState *estate,
ExprContext *econtext, bool *isnull,
Datum caseval_datum,
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index bae466b523..6bdd9f5121 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -69,8 +69,10 @@ typedef enum
/* Convenience macros */
#define DatumGetJsonbP(d) ((Jsonb *) PG_DETOAST_DATUM(d))
+#define DatumGetJsonbValueP(d) ((JsonbValue *) DatumGetPointer(d))
#define DatumGetJsonbPCopy(d) ((Jsonb *) PG_DETOAST_DATUM_COPY(d))
#define JsonbPGetDatum(p) PointerGetDatum(p)
+#define JsonbValuePGetDatum(p) PointerGetDatum(p)
#define PG_GETARG_JSONB_P(x) DatumGetJsonbP(PG_GETARG_DATUM(x))
#define PG_GETARG_JSONB_P_COPY(x) DatumGetJsonbPCopy(PG_GETARG_DATUM(x))
#define PG_RETURN_JSONB_P(x) PG_RETURN_POINTER(x)
--
2.35.3
v2-0001-in-JsonExprState-just-store-a-pointer-to-the-inpu.patch (application/octet-stream, 2.2K) download
From b875ed56e8d991927cdc320d6658e247af005417 Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Wed, 22 Jun 2022 12:00:47 -0400
Subject: [PATCH v2 1/3] in JsonExprState just store a pointer to the input
FmgrInfo
---
src/backend/executor/execExpr.c | 5 ++++-
src/backend/executor/execExprInterp.c | 2 +-
src/include/executor/execExpr.h | 2 +-
3 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index c8d7145fe3..a55e5000e2 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2606,11 +2606,14 @@ ExecInitExprRec(Expr *node, ExprState *state,
(jexpr->result_coercion && jexpr->result_coercion->via_io))
{
Oid typinput;
+ FmgrInfo *finfo;
/* lookup the result type's input function */
getTypeInputInfo(jexpr->returning->typid, &typinput,
&jsestate->input.typioparam);
- fmgr_info(typinput, &jsestate->input.func);
+ finfo = palloc0(sizeof(FmgrInfo));
+ fmgr_info(typinput, finfo);
+ jsestate->input.finfo = finfo;
}
jsestate->args = NIL;
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 723770fda0..0512a81c7c 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -4664,7 +4664,7 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
/* strip quotes and call typinput function */
char *str = *isNull ? NULL : JsonbUnquote(jb);
- return InputFunctionCall(&jsestate->input.func, str,
+ return InputFunctionCall(jsestate->input.finfo, str,
jsestate->input.typioparam,
jexpr->returning->typmod);
}
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 1e3f1bbee8..e55a572854 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -754,7 +754,7 @@ typedef struct JsonExprState
struct
{
- FmgrInfo func; /* typinput function for output type */
+ FmgrInfo *finfo; /* typinput function for output type */
Oid typioparam;
} input; /* I/O info for output type */
--
2.35.3
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-07-19 15:37 ` Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-07-19 15:37 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-07-19 20:40:11 +0900, Amit Langote wrote:
> About that, I was wondering if the blocks in llvm_compile_expr() need
> to be hand-coded to match what's added in ExecInterpExpr() or if I've
> missed some tool that can be used instead?
The easiest way is to just call an external function for the implementation of
the step. But yes, otherwise you need to handcraft it.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-07-20 14:09 ` Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Amit Langote @ 2022-07-20 14:09 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On Wed, Jul 20, 2022 at 12:37 AM Andres Freund <andres@anarazel.de> wrote:
> On 2022-07-19 20:40:11 +0900, Amit Langote wrote:
> > About that, I was wondering if the blocks in llvm_compile_expr() need
> > to be hand-coded to match what's added in ExecInterpExpr() or if I've
> > missed some tool that can be used instead?
>
> The easiest way is to just call an external function for the implementation of
> the step. But yes, otherwise you need to handcraft it.
Ok, thanks.
So I started updating llvm_compile_expr() for handling the new
ExprEvalSteps that the patch adds to ExecExprInterp(), but quickly
realized that code could have been consolidated into less code, or
IOW, into fewer new ExprEvalSteps. So, I refactored things that way
and am now retrying adding the code to llvm_compile_expr() based on
new, better consolidated, code.
Here's the updated version, without the llvm pieces, in case you'd
like to look at it even in this state. I'll post a version with llvm
pieces filled in tomorrow. (I have merged the different patches into
one for convenience.)
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
v3-0001-Some-improvements-to-JsonExpr-evaluation.patch (application/octet-stream, 45.2K) download
From c0d4ead7c31919ebb9147275b27685ca1f556698 Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Wed, 22 Jun 2022 12:00:47 -0400
Subject: [PATCH v3] Some improvements to JsonExpr evaluation
* Store a pointer to the returning type input function FmgrInfo,
not the struct itself, in JsonExprState
* Evaluate various JsonExpr sub-expressions using parent ExprState
These include PASSING args, ON ERROR, ON EMPTY default expressions.
PASSING args are simple, because they all must be evaluated before
the JSON path evaluation can occur anyway, so pushing those
expressions to be uncondionally evaluated before the step to
evaluate JsonExpr proper suffices. Evaluation of the ON ERROR and
the ON EMPTY expressions is conditional on what JSON item pops out
of the path evaluation (ExecEvalJson()), so needs more logic to
gate the steps for their expressions that are pushed into the parent
ExprState, rather than be imlemented using separate ExprStates.
To that end, this moves the ON ERROR, ON EMPTY behavior decision logic
out of ExecEvalJson() and ExecEvalJsonExpr() into a new function
ExecEvalJsonExprBehavior().
* Final result coercion is now also computed as a separate step that
gets pushed into the parent ExprState. Though given that any
coercion evaluation errors must be handled the same way as the errors
of JSON item evaluation, that is, to be handled according the ON ERROR
behavior specification, a sub-transaction must be used around coercion
evaluation, so the ExprState to use for the same must have to be a
different one from the JsonExpr's. However, instead of using one
ExprState for each JsonCoercion member of JsonItemCoercions, only one
is used for the whole JsonItemCoercions. The individual JsonCoercion
members (or their Exprs) are handled by steps pushed into that
ExprState, of which only the one that applies to a given JSON item is
chosen at the runtime and others "jumped" over.
TODO: update llvm_compile_expr()
---
src/backend/executor/execExpr.c | 377 ++++++++++++++++++----
src/backend/executor/execExprInterp.c | 431 ++++++++++++++++++--------
src/backend/jit/llvm/llvmjit_expr.c | 31 +-
src/backend/jit/llvm/llvmjit_types.c | 1 +
src/backend/optimizer/util/clauses.c | 6 +-
src/include/executor/execExpr.h | 176 ++++++++---
src/include/utils/jsonb.h | 2 +
7 files changed, 792 insertions(+), 232 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index c8d7145fe3..d7cda5b850 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2558,113 +2558,356 @@ ExecInitExprRec(Expr *node, ExprState *state,
break;
}
+ case T_JsonCoercion:
+ {
+ JsonCoercion *coercion = castNode(JsonCoercion, node);
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
+
+ /*
+ * Only ever get here for JsonExpr.result_expression and the
+ * caller should have checked this.
+ */
+ Assert(coercion->expr);
+
+ /*
+ * First push a step to read the value provided by the parent
+ * JsonExpr via a CaseTestExpr.
+ */
+ scratch.opcode = EEOP_CASE_TESTVAL;
+ scratch.d.casetest.value = state->innermost_caseval;
+ scratch.d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Push step(s) to compute coercion->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
+
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) coercion->expr, state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
+
+ break;
+ }
+
case T_JsonExpr:
{
JsonExpr *jexpr = castNode(JsonExpr, node);
JsonExprState *jsestate = palloc0(sizeof(JsonExprState));
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
ListCell *argexprlc;
ListCell *argnamelc;
+ int skip_step_off;
+ int behavior_step_off;
+ int onempty_default_step_off;
+ int onerror_default_step_off;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
+ int coercion_step_off;
+ ExprEvalStep *as;
+ bool throwErrors =
+ (jexpr->on_error->btype == JSON_BEHAVIOR_ERROR);
+
+ jsestate->jsexpr = jexpr;
+
+ /*
+ * Add steps to compute formatted_expr, pathspec, and
+ * PASSING arg expressions as things that must be
+ * evaluated before the "main" JSON path evaluation.
+ */
+ ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
+ &pre_eval->formatted_expr.value,
+ &pre_eval->formatted_expr.isnull);
+ ExecInitExprRec((Expr *) jexpr->path_spec, state,
+ &pre_eval->pathspec.value,
+ &pre_eval->pathspec.isnull);
- scratch.opcode = EEOP_JSONEXPR;
+ /*
+ * However, before pushing steps for PASSING args, push a step
+ * that will decide whether to skip evaluating the args and
+ * JSON path evaluation depending on whether either of
+ * formatted_expr and pathspec is NULL.
+ */
+ scratch.opcode = EEOP_JSONEXPR_SKIP;
+ scratch.d.jsonexpr_skip.jsestate = jsestate;
+ skip_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ jsestate->pre_eval.args = NIL;
+ forboth(argexprlc, jexpr->passing_values,
+ argnamelc, jexpr->passing_names)
+ {
+ Expr *argexpr = (Expr *) lfirst(argexprlc);
+ String *argname = lfirst_node(String, argnamelc);
+ JsonPathVariableEvalContext *var = palloc(sizeof(*var));
+
+ var->name = pstrdup(argname->sval);
+ var->typid = exprType((Node *) argexpr);
+ var->typmod = exprTypmod((Node *) argexpr);
+
+ /*
+ * Not necessary when being evaluated for a JsonExpr, like
+ * in this case.
+ */
+ var->estate = NULL;
+ var->econtext = NULL;
+ var->mcxt = NULL;
+
+ /*
+ * Mark these as always evaluated because they must have
+ * been evaluated before JSON path evaluation begins,
+ * because we haven't pushed the step for the latter yet.
+ */
+ var->evaluated = true;
+
+ ExecInitExprRec((Expr *) argexpr, state,
+ &var->value,
+ &var->isnull);
+
+ pre_eval->args = lappend(pre_eval->args, var);
+ }
+
+ /* Now push the step for the actual JSON path evaluation. */
+ scratch.opcode = EEOP_JSONEXPR_PATH;
scratch.d.jsonexpr.jsestate = jsestate;
+ ExprEvalPushStep(state, &scratch);
- jsestate->jsexpr = jexpr;
+ /*
+ * Now push steps to control the expressions evaluated based
+ * on the result of JSON path evaluation.
+ */
- jsestate->formatted_expr =
- palloc(sizeof(*jsestate->formatted_expr));
+ /*
+ * Step to handle ON ERROR and ON EMPTY behavior correctly.
+ */
+ scratch.opcode = EEOP_JSONEXPR_BEHAVIOR;
+ scratch.d.jsonexpr_behavior.jsestate = jsestate;
+ behavior_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
- ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
- &jsestate->formatted_expr->value,
- &jsestate->formatted_expr->isnull);
+ onempty_default_step_off = state->steps_len;
+ if (jexpr->on_empty)
+ {
+ /* Push step(s) for the default ON EMPTY expression. */
+ if (jexpr->on_empty->default_expr)
+ {
+ ExecInitExprRec((Expr *) jexpr->on_empty->default_expr,
+ state, resv, resnull);
- jsestate->pathspec =
- palloc(sizeof(*jsestate->pathspec));
+ /* Emit JUMP step to jump to end of JsonExpr code */
+ scratch.opcode = EEOP_JUMP;
+ scratch.d.jump.jumpdone = -1; /* computed later */
+ ExprEvalPushStep(state, &scratch);
- ExecInitExprRec((Expr *) jexpr->path_spec, state,
- &jsestate->pathspec->value,
- &jsestate->pathspec->isnull);
+ /*
+ * Don't know address for that jump yet, compute once the
+ * whole JsonExpr is built.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len - 1);
+ }
+ }
- jsestate->res_expr =
- palloc(sizeof(*jsestate->res_expr));
+ onerror_default_step_off = state->steps_len;
+ if (jexpr->on_error)
+ {
+ /* Push step(s) for the default ON ERROR expression. */
+ if (jexpr->on_error->default_expr)
+ {
+ ExecInitExprRec((Expr *) jexpr->on_error->default_expr,
+ state, resv, resnull);
- jsestate->result_expr = jexpr->result_coercion
- ? ExecInitExprWithCaseValue((Expr *) jexpr->result_coercion->expr,
- state->parent,
- &jsestate->res_expr->value,
- &jsestate->res_expr->isnull)
- : NULL;
+ /* Emit JUMP step to jump to end of JsonExpr code */
+ scratch.opcode = EEOP_JUMP;
+ scratch.d.jump.jumpdone = -1; /* computed later */
+ ExprEvalPushStep(state, &scratch);
- jsestate->default_on_empty = !jexpr->on_empty ? NULL :
- ExecInitExpr((Expr *) jexpr->on_empty->default_expr,
- state->parent);
+ /*
+ * Don't know address for that jump yet, compute once the
+ * whole JsonExpr is built.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len - 1);
+ }
+ }
+
+ /*
+ * Step to handle applying coercion correctly. Might be
+ * skipped if no coercion needs to be applied separately
+ * necessary to do so will be set later.
+ */
+ scratch.opcode = EEOP_JSONEXPR_COERCION;
+ scratch.d.jsonexpr_coercion.jsestate = jsestate;
+ coercion_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ /*
+ * Adjust jump target addresses in various post-eval steps now
+ * that we have all the steps in place.
+ */
+
+ /* EEOP_JSONEXPR_SKIP */
+ as = &state->steps[skip_step_off];
+ as->d.jsonexpr_skip.jump_coercion = coercion_step_off;
- jsestate->default_on_error =
- ExecInitExpr((Expr *) jexpr->on_error->default_expr,
- state->parent);
+ /* EEOP_JSONEXPR_BEHAVIOR */
+ as = &state->steps[behavior_step_off];
+ as->d.jsonexpr_behavior.jump_onerror_default = onerror_default_step_off;
+ as->d.jsonexpr_behavior.jump_onempty_default = onempty_default_step_off;
+ as->d.jsonexpr_behavior.jump_coercion = coercion_step_off;
+ as->d.jsonexpr_behavior.jump_skip_coercion = coercion_step_off + 1;
+ /* EEOP_JSONEXPR_COERCION */
+ as = &state->steps[coercion_step_off];
+ as->d.jsonexpr_coercion.jump_coercion_error = behavior_step_off;
+
+ foreach(lc, adjust_jumps)
+ {
+ as = &state->steps[lfirst_int(lc)];
+
+ as->d.jump.jumpdone = state->steps_len;
+ }
+
+ /*
+ * Set if we must use a sub-transaction around path evaluation
+ * that can be aborted on error instead of aborting the parent
+ * query.
+ */
+ jsestate->needSubtrans =
+ ExecEvalJsonNeedsSubTransaction(jexpr,
+ true /* coerce */);
+ jsestate->coercionNeedSubtrans =
+ (!jsestate->needSubtrans && !throwErrors);
+
+ /*
+ * Set RETURNING type's input function used by
+ * ExecEvalJsonExprCoercion().
+ */
if (jexpr->omit_quotes ||
(jexpr->result_coercion && jexpr->result_coercion->via_io))
{
Oid typinput;
+ FmgrInfo *finfo;
/* lookup the result type's input function */
getTypeInputInfo(jexpr->returning->typid, &typinput,
&jsestate->input.typioparam);
- fmgr_info(typinput, &jsestate->input.func);
+ finfo = palloc0(sizeof(FmgrInfo));
+ fmgr_info(typinput, finfo);
+ jsestate->input.finfo = finfo;
}
- jsestate->args = NIL;
+ if (jexpr->result_coercion && jexpr->result_coercion->expr)
+ jsestate->result_coercion =
+ ExecInitExprWithCaseValue((Expr *)
+ jexpr->result_coercion,
+ state->parent,
+ resv, resnull);
- forboth(argexprlc, jexpr->passing_values,
- argnamelc, jexpr->passing_names)
- {
- Expr *argexpr = (Expr *) lfirst(argexprlc);
- String *argname = lfirst_node(String, argnamelc);
- JsonPathVariableEvalContext *var = palloc(sizeof(*var));
+ /* Set up coercion related data structures. */
+ if (jexpr->coercions)
+ jsestate->coercions =
+ ExecInitExprWithCaseValue((Expr *) jexpr->coercions,
+ state->parent,
+ resv, resnull);
- var->name = pstrdup(argname->sval);
- var->typid = exprType((Node *) argexpr);
- var->typmod = exprTypmod((Node *) argexpr);
- var->estate = ExecInitExpr(argexpr, state->parent);
- var->econtext = NULL;
- var->mcxt = NULL;
- var->evaluated = false;
- var->value = (Datum) 0;
- var->isnull = true;
+ break;
+ }
- jsestate->args =
- lappend(jsestate->args, var);
- }
+ case T_JsonItemCoercions:
+ {
+ JsonItemCoercions *coercions = castNode(JsonItemCoercions,
+ node);
+ JsonItemCoercionsState *jcstate =
+ palloc0(sizeof(JsonItemCoercionsState));
+ JsonCoercion **coercion;
+ JsonItemCoercionState *cstate;
+ ExprEvalStep *as;
+ int json_item_coercion_step_id;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
- jsestate->cache = NULL;
+ /*
+ * First push a step to read the value provided by the parent
+ * JsonExpr via a CaseTestExpr.
+ */
+ scratch.opcode = EEOP_CASE_TESTVAL;
+ scratch.d.casetest.value = state->innermost_caseval;
+ scratch.d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, &scratch);
- if (jexpr->coercions)
+ /* Push the control step. */
+ scratch.opcode = EEOP_JSON_ITEM_COERCION;
+ scratch.d.json_item_coercion.jcstate = jcstate;
+ json_item_coercion_step_id = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+ /* Will set jump_skip_coercion target address below. */
+
+ /*
+ * Now push the steps of individual coercion's expression, if
+ * needed.
+ */
+ for (cstate = &jcstate->null,
+ coercion = &coercions->null;
+ coercion <= &coercions->composite;
+ coercion++, cstate++)
{
- JsonCoercion **coercion;
- struct JsonCoercionState *cstate;
- Datum *caseval;
- bool *casenull;
+ cstate->coercion = *coercion;
+ if (cstate->coercion && cstate->coercion->expr)
+ {
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
- jsestate->coercion_expr =
- palloc(sizeof(*jsestate->coercion_expr));
+ cstate->jump_eval_expr = state->steps_len;
- caseval = &jsestate->coercion_expr->value;
- casenull = &jsestate->coercion_expr->isnull;
+ /* Push step(s) to compute (*coercion)->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
- for (cstate = &jsestate->coercions.null,
- coercion = &jexpr->coercions->null;
- coercion <= &jexpr->coercions->composite;
- coercion++, cstate++)
- {
- cstate->coercion = *coercion;
- cstate->estate = *coercion ?
- ExecInitExprWithCaseValue((Expr *) (*coercion)->expr,
- state->parent,
- caseval, casenull) : NULL;
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) cstate->coercion->expr,
+ state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
}
+ else
+ cstate->jump_eval_expr = -1;
+
+ /* Emit JUMP step to jump to end of coercions code */
+ scratch.opcode = EEOP_JUMP;
+
+ /*
+ * Remember JUMP step address to set the actual jump
+ * target address below.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len);
+ ExprEvalPushStep(state, &scratch);
+ }
+
+ /*
+ * Adjust the jump target address of the control step and all
+ * the jumps we added in the above loop to make them point to
+ * the step after the last step that would have been added
+ * above.
+ */
+ as = &state->steps[json_item_coercion_step_id];
+ as->d.json_item_coercion.jump_skip_item_coercion = state->steps_len;
+ foreach(lc, adjust_jumps)
+ {
+ int jump_step_id = lfirst_int(lc);
+
+ as = &state->steps[jump_step_id];
+ as->d.jump.jumpdone = state->steps_len;
}
- ExprEvalPushStep(state, &scratch);
break;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 723770fda0..02bfed84fc 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -158,6 +158,15 @@ static TupleDesc get_cached_rowtype(Oid type_id, int32 typmod,
bool *changed);
static void ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, bool checkisnull);
+static Datum ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null);
+typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
+ Datum item, bool *resnull, void *p, bool *error);
+static Datum ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
+ ExprContext *econtext,
+ Datum res, bool *resnull,
+ void *p, bool *error, bool subtrans);
+static Datum ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
+ Datum res, bool *isNull, void *p, bool *error);
/* fast-path evaluation functions */
static Datum ExecJustInnerVar(ExprState *state, ExprContext *econtext, bool *isnull);
@@ -490,7 +499,11 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SUBPLAN,
&&CASE_EEOP_JSON_CONSTRUCTOR,
&&CASE_EEOP_IS_JSON,
- &&CASE_EEOP_JSONEXPR,
+ &&CASE_EEOP_JSONEXPR_SKIP,
+ &&CASE_EEOP_JSONEXPR_PATH,
+ &&CASE_EEOP_JSONEXPR_BEHAVIOR,
+ &&CASE_EEOP_JSONEXPR_COERCION,
+ &&CASE_EEOP_JSON_ITEM_COERCION,
&&CASE_EEOP_AGG_STRICT_DESERIALIZE,
&&CASE_EEOP_AGG_DESERIALIZE,
&&CASE_EEOP_AGG_STRICT_INPUT_CHECK_ARGS,
@@ -1817,13 +1830,68 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
- EEO_CASE(EEOP_JSONEXPR)
+ EEO_CASE(EEOP_JSONEXPR_SKIP)
+ {
+ /*
+ * Skip JSON evaluation if either of the input expressions has
+ * turned out to be NULL, though do execute domain checks for
+ * NULLs, which are handled by the coercion step.
+ */
+ if (ExecEvalJsonSkip(op))
+ EEO_JUMP(op->d.jsonexpr_skip.jump_coercion);
+
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_PATH)
{
/* too complex for an inline implementation */
ExecEvalJson(state, op, econtext);
EEO_NEXT();
}
+ EEO_CASE(EEOP_JSONEXPR_BEHAVIOR)
+ {
+ int jumpaddr = ExecEvalJsonExprBehavior(op);
+
+ EEO_JUMP(jumpaddr);
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_COERCION)
+ {
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+
+ post_eval->coercion_error = false;
+ *op->resvalue =
+ ExecEvalJsonExprSubtrans(ExecEvalJsonExprCoercion, op, econtext,
+ *op->resvalue,
+ op->resnull, NULL,
+ &post_eval->coercion_error,
+ post_eval->coercion_use_subtrans);
+ if (post_eval->coercion_error)
+ EEO_JUMP(op->d.jsonexpr_coercion.jump_coercion_error);
+
+ EEO_NEXT();
+ }
+
+ EEO_CASE(EEOP_JSON_ITEM_COERCION)
+ {
+ JsonItemCoercionsState *jcstate = op->d.json_item_coercion.jcstate;
+ JsonItemCoercionState *cstate = NULL;
+
+ *op->resvalue =
+ ExecPrepareJsonItemCoercion(*op->resvalue, jcstate, &cstate);
+ if (cstate->coercion && cstate->coercion->expr)
+ {
+ Assert(cstate->jump_eval_expr >= 0);
+ EEO_JUMP(cstate->jump_eval_expr);
+ }
+
+ /* Skip over all of the steps added for this JsonItemCoercions. */
+ EEO_JUMP(op->d.json_item_coercion.jump_skip_item_coercion);
+ }
+
EEO_CASE(EEOP_LAST)
{
/* unreachable */
@@ -4602,8 +4670,7 @@ ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
* Evaluate a JSON error/empty behavior result.
*/
static Datum
-ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
- ExprState *default_estate, bool *is_null)
+ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null)
{
*is_null = false;
@@ -4628,7 +4695,9 @@ ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
return (Datum) 0;
case JSON_BEHAVIOR_DEFAULT:
- return ExecEvalExpr(default_estate, econtext, is_null);
+ /* Always handled in the caller. */
+ Assert(false);
+ return (Datum) 0;
default:
elog(ERROR, "unrecognized SQL/JSON behavior %d", behavior->btype);
@@ -4643,18 +4712,17 @@ static Datum
ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
Datum res, bool *isNull, void *p, bool *error)
{
- ExprState *estate = p;
- JsonExprState *jsestate;
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ JsonExpr *jexpr = jsestate->jsexpr;
+ ExprState *estate = post_eval->coercions;
if (estate) /* coerce using specified expression */
return ExecEvalExpr(estate, econtext, isNull);
- jsestate = op->d.jsonexpr.jsestate;
-
- if (jsestate->jsexpr->op != JSON_EXISTS_OP)
+ if (jexpr->op != JSON_EXISTS_OP)
{
- JsonCoercion *coercion = jsestate->jsexpr->result_coercion;
- JsonExpr *jexpr = jsestate->jsexpr;
+ JsonCoercion *coercion = jexpr->result_coercion;
Jsonb *jb = *isNull ? NULL : DatumGetJsonbP(res);
if ((coercion && coercion->via_io) ||
@@ -4664,7 +4732,7 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
/* strip quotes and call typinput function */
char *str = *isNull ? NULL : JsonbUnquote(jb);
- return InputFunctionCall(&jsestate->input.func, str,
+ return InputFunctionCall(jsestate->input.finfo, str,
jsestate->input.typioparam,
jexpr->returning->typmod);
}
@@ -4672,17 +4740,20 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
return json_populate_type(res, JSONBOID,
jexpr->returning->typid,
jexpr->returning->typmod,
- &jsestate->cache,
+ &post_eval->cache,
econtext->ecxt_per_query_memory,
isNull);
}
- if (jsestate->result_expr)
+ /*
+ * Let the caller know that no coercion was done here, so it can
+ * coerce with jexpr->result_coercion if there's one.
+ */
+ if (jsestate->result_coercion)
{
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *isNull;
-
- res = ExecEvalExpr(jsestate->result_expr, econtext, isNull);
+ *op->resvalue = res;
+ *op->resnull = *isNull;
+ return ExecEvalExpr(jsestate->result_coercion, econtext, isNull);
}
return res;
@@ -4717,17 +4788,30 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
if (!var)
return -1;
- if (!var->evaluated)
+ /*
+ * When belonging to a JsonExpr, path variables are computed with the
+ * JsonExpr's ExprState (var->estate is NULL), so don't need to be computed
+ * here. In some other cases, such as when the path variables belonging
+ * to a JsonTable instead, those variables must be evaluated on their own,
+ * without the enclosing JsonExpr itself needing to be evaluated, so must
+ * be handled here.
+ */
+ if (var->estate && !var->evaluated)
{
MemoryContext oldcxt = var->mcxt ?
MemoryContextSwitchTo(var->mcxt) : NULL;
+ Assert(var->econtext != NULL);
var->value = ExecEvalExpr(var->estate, var->econtext, &var->isnull);
var->evaluated = true;
if (oldcxt)
MemoryContextSwitchTo(oldcxt);
}
+ else
+ {
+ Assert(var->evaluated);
+ }
if (var->isnull)
{
@@ -4741,19 +4825,84 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
return id;
}
+/*
+ * Return a coercion among those given in 'coercions' for given
+ * JSON item.
+ */
+JsonCoercion *
+ExecGetJsonItemCoercion(JsonbValue *item, JsonItemCoercions *coercions)
+{
+ if (item->type == jbvBinary &&
+ JsonContainerIsScalar(item->val.binary.data))
+ {
+ JsonbValue buf;
+ bool res PG_USED_FOR_ASSERTS_ONLY;
+
+ res = JsonbExtractScalar(item->val.binary.data, &buf);
+ item = &buf;
+ Assert(res);
+ }
+
+ /* get coercion state reference and datum of the corresponding SQL type */
+ switch (item->type)
+ {
+ case jbvNull:
+ return coercions->null;
+
+ case jbvString:
+ return coercions->string;
+
+ case jbvNumeric:
+ return coercions->numeric;
+
+ case jbvBool:
+ return coercions->boolean;
+
+ case jbvDatetime:
+ switch (item->val.datetime.typid)
+ {
+ case DATEOID:
+ return coercions->date;
+ case TIMEOID:
+ return coercions->time;
+ case TIMETZOID:
+ return coercions->timetz;
+ case TIMESTAMPOID:
+ return coercions->timestamp;
+ case TIMESTAMPTZOID:
+ return coercions->timestamptz;
+ default:
+ elog(ERROR, "unexpected jsonb datetime type oid %u",
+ item->val.datetime.typid);
+ }
+ break;
+
+ case jbvArray:
+ case jbvObject:
+ case jbvBinary:
+ return coercions->composite;
+
+ default:
+ elog(ERROR, "unexpected jsonb value type %d", item->type);
+ }
+
+ Assert(false);
+ return NULL;
+}
+
/*
* Prepare SQL/JSON item coercion to the output type. Returned a datum of the
* corresponding SQL type and a pointer to the coercion state.
*/
Datum
-ExecPrepareJsonItemCoercion(JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pcoercion)
+ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pcoercion)
{
- struct JsonCoercionState *coercion;
+ JsonbValue *item = DatumGetJsonbValueP(itemval);
+ JsonItemCoercionState *coercion;
Datum res;
- JsonbValue buf;
+ JsonbValue buf;
if (item->type == jbvBinary &&
JsonContainerIsScalar(item->val.binary.data))
@@ -4832,9 +4981,6 @@ ExecPrepareJsonItemCoercion(JsonbValue *item,
return res;
}
-typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
- Datum item, bool *resnull, void *p, bool *error);
-
static Datum
ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
ExprContext *econtext,
@@ -4905,7 +5051,6 @@ typedef struct
{
JsonPath *path;
bool *error;
- bool coercionInSubtrans;
} ExecEvalJsonExprContext;
static Datum
@@ -4916,16 +5061,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
ExecEvalJsonExprContext *cxt = pcxt;
JsonPath *path = cxt->path;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
- ExprState *estate = NULL;
- bool empty = false;
+ bool *empty = &post_eval->empty;
Datum res = (Datum) 0;
switch (jexpr->op)
{
case JSON_QUERY_OP:
- res = JsonPathQuery(item, path, jexpr->wrapper, &empty, error,
- jsestate->args);
+ res = JsonPathQuery(item, path, jexpr->wrapper, empty, error,
+ pre_eval->args);
if (error && *error)
{
*resnull = true;
@@ -4936,17 +5082,20 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
case JSON_VALUE_OP:
{
- struct JsonCoercionState *jcstate;
- JsonbValue *jbv = JsonPathValue(item, path, &empty, error,
- jsestate->args);
+ JsonCoercion *coercion;
+ JsonbValue *jbv = JsonPathValue(item, path, empty, error,
+ pre_eval->args);
if (error && *error)
+ {
+ *resnull = true;
return (Datum) 0;
+ }
if (!jbv) /* NULL or empty */
break;
- Assert(!empty);
+ Assert(!*empty);
*resnull = false;
@@ -4959,19 +5108,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
break;
}
- /* Use coercion from SQL/JSON item type to the output type */
- res = ExecPrepareJsonItemCoercion(jbv,
- jsestate->jsexpr->returning,
- &jsestate->coercions,
- &jcstate);
-
- if (jcstate->coercion &&
- (jcstate->coercion->via_io ||
- jcstate->coercion->via_populate))
+ /*
+ * Error out no cast exists to coerce SQL/JSON item to the
+ * the output type
+ */
+ coercion = ExecGetJsonItemCoercion(jbv, jsestate->jsexpr->coercions);
+ if (coercion && (coercion->via_io || coercion->via_populate))
{
if (error)
{
*error = true;
+ *resnull = true;
return (Datum) 0;
}
@@ -4983,32 +5130,30 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_SQL_JSON_ITEM_CANNOT_BE_CAST_TO_TARGET_TYPE),
errmsg("SQL/JSON item cannot be cast to target type")));
}
- else if (!jcstate->estate)
- return res; /* no coercion */
-
- /* coerce using specific expression */
- estate = jcstate->estate;
- jsestate->coercion_expr->value = res;
- jsestate->coercion_expr->isnull = *resnull;
+ else
+ {
+ /* Coerce using specific expression. */
+ res = JsonbValuePGetDatum(jbv);
+ post_eval->coercions = jsestate->coercions;
+ }
break;
}
case JSON_EXISTS_OP:
{
bool exists = JsonPathExists(item, path,
- jsestate->args,
+ pre_eval->args,
error);
*resnull = error && *error;
res = BoolGetDatum(exists);
- if (!jsestate->result_expr)
+ if (jexpr->result_coercion == NULL)
+ {
+ /* No coercion needed */
+ post_eval->coercion_done = true;
return res;
-
- /* coerce using result expression */
- estate = jsestate->result_expr;
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *resnull;
+ }
break;
}
@@ -5021,7 +5166,11 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
return (Datum) 0;
}
- if (empty)
+ /*
+ * If the ON EMPTY behavior is to cause an error, do so here. Other
+ * behaviors will be handled by the caller.
+ */
+ if (*empty)
{
Assert(jexpr->on_empty); /* it is not JSON_EXISTS */
@@ -5037,29 +5186,13 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_NO_SQL_JSON_ITEM),
errmsg("no SQL/JSON item")));
}
-
- if (jexpr->on_empty->btype == JSON_BEHAVIOR_DEFAULT)
-
- /*
- * Execute DEFAULT expression as a coercion expression, because
- * its result is already coerced to the target type.
- */
- estate = jsestate->default_on_empty;
- else
- /* Execute ON EMPTY behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_empty,
- jsestate->default_on_empty,
- resnull);
}
- return ExecEvalJsonExprSubtrans(ExecEvalJsonExprCoercion, op, econtext,
- res, resnull, estate, error,
- cxt->coercionInSubtrans);
+ return res;
}
bool
-ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *coercions)
+ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce)
{
if (jsexpr->on_error->btype == JSON_BEHAVIOR_ERROR)
return false;
@@ -5067,12 +5200,96 @@ ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
if (jsexpr->op == JSON_EXISTS_OP && !jsexpr->result_coercion)
return false;
- if (!coercions)
+ if (!coerce)
+ return true;
+
+ return false;
+}
+
+/* Skip calling ExecEvalJson() on a JsonExpr? */
+bool
+ExecEvalJsonSkip(ExprEvalStep *op)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_skip.jsestate;
+
+ if (jsestate->pre_eval.formatted_expr.isnull ||
+ jsestate->pre_eval.pathspec.isnull)
+ {
+ *op->resvalue = 0;
+ *op->resnull = true;
+ Assert(jsestate->post_eval.coercions == NULL);
+ /* A signal to the coercion step to not use a sub-trancaction. */
+ jsestate->post_eval.coercion_error = true;
return true;
+ }
return false;
}
+int
+ExecEvalJsonExprBehavior(ExprEvalStep *op)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_behavior.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ JsonBehavior *behavior = NULL;
+ int jump_to = -1;
+ bool error = (post_eval->error || post_eval->coercion_error);
+
+ /*
+ * Don't use a sub-transaction if coercing the ON ERROR expression, so
+ * that any errors encountered downstream this time are thrown there,
+ * that is, not rethrown to be handled by the caller.
+ *
+ * XXX - doesn't that violate ON ERROR behavior? Actually, that is
+ * how it behaved even before this expression eval logic rewrite!
+ */
+ post_eval->coercion_use_subtrans =
+ (jsestate->coercionNeedSubtrans && !error);
+
+ /*
+ * Directly go to the coercion step to coerce JSON item as is, os skip
+ * the coercion step if JSON item is already coerced by ExecEvalJson().
+ */
+ if (!error && !post_eval->empty)
+ return !post_eval->coercion_done ?
+ op->d.jsonexpr_behavior.jump_coercion :
+ op->d.jsonexpr_behavior.jump_skip_coercion;
+
+ if (error)
+ {
+ behavior = jsestate->jsexpr->on_error;
+ jump_to = op->d.jsonexpr_behavior.jump_onerror_default;
+ }
+ else if (post_eval->empty)
+ {
+ behavior = jsestate->jsexpr->on_empty;
+ jump_to = op->d.jsonexpr_behavior.jump_onempty_default;
+ }
+
+ Assert(behavior && jump_to >= 0);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate
+ * value and go to the coercion step.
+ */
+ if (behavior->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ *op->resvalue = ExecEvalJsonBehavior(behavior, op->resnull);
+
+ if (post_eval->coercion_error)
+ {
+ post_eval->coercion_done = false;
+ post_eval->coercions = NULL;
+ }
+
+ return op->d.jsonexpr_behavior.jump_coercion;
+ }
+
+ /* Else evaluate the default ON ERROR or ON EMPTY expression. */
+
+ return jump_to;
+}
+
/* ----------------------------------------------------------------
* ExecEvalJson
* ----------------------------------------------------------------
@@ -5082,64 +5299,30 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
{
ExecEvalJsonExprContext cxt;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
Datum item;
Datum res = (Datum) 0;
JsonPath *path;
- ListCell *lc;
- bool error = false;
- bool needSubtrans;
bool throwErrors = jexpr->on_error->btype == JSON_BEHAVIOR_ERROR;
*op->resnull = true; /* until we get a result */
*op->resvalue = (Datum) 0;
- if (jsestate->formatted_expr->isnull || jsestate->pathspec->isnull)
- {
- /* execute domain checks for NULLs */
- (void) ExecEvalJsonExprCoercion(op, econtext, res, op->resnull,
- NULL, NULL);
-
- Assert(*op->resnull);
- return;
- }
-
- item = jsestate->formatted_expr->value;
- path = DatumGetJsonPathP(jsestate->pathspec->value);
-
- /* reset JSON path variable contexts */
- foreach(lc, jsestate->args)
- {
- JsonPathVariableEvalContext *var = lfirst(lc);
-
- var->econtext = econtext;
- var->evaluated = false;
- }
+ item = pre_eval->formatted_expr.value;
+ path = DatumGetJsonPathP(pre_eval->pathspec.value);
- needSubtrans = ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ /* Reset JsonExprPostEvalState for this evaluation. */
+ memset(post_eval, 0, sizeof(*post_eval));
cxt.path = path;
- cxt.error = throwErrors ? NULL : &error;
- cxt.coercionInSubtrans = !needSubtrans && !throwErrors;
- Assert(!needSubtrans || cxt.error);
+ cxt.error = throwErrors ? NULL : &post_eval->error;
+ Assert(!jsestate->needSubtrans || cxt.error);
res = ExecEvalJsonExprSubtrans(ExecEvalJsonExpr, op, econtext, item,
op->resnull, &cxt, cxt.error,
- needSubtrans);
-
- if (error)
- {
- /* Execute ON ERROR behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_error,
- jsestate->default_on_error,
- op->resnull);
-
- /* result is already coerced in DEFAULT behavior case */
- if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
- res = ExecEvalJsonExprCoercion(op, econtext, res,
- op->resnull,
- NULL, NULL);
- }
+ jsestate->needSubtrans);
*op->resvalue = res;
}
diff --git a/src/backend/jit/llvm/llvmjit_expr.c b/src/backend/jit/llvm/llvmjit_expr.c
index b6b6512ef1..b96c77a3ab 100644
--- a/src/backend/jit/llvm/llvmjit_expr.c
+++ b/src/backend/jit/llvm/llvmjit_expr.c
@@ -2359,12 +2359,39 @@ llvm_compile_expr(ExprState *state)
LLVMBuildBr(b, opblocks[opno + 1]);
break;
- case EEOP_JSONEXPR:
+ case EEOP_JSONEXPR_SKIP:
+ {
+ int jumpaddr = op->d.jsonexpr_skip.jump_coercion;
+ LLVMValueRef v_params[1];
+ LLVMValueRef v_ret;
+
+ v_params[0] = l_ptr_const(op, l_ptr(StructExprEvalStep));
+ v_ret = LLVMBuildCall(b,
+ llvm_pg_func(mod, "ExecEvalJsonSkip"),
+ v_params, lengthof(v_params), "");
+ v_ret = LLVMBuildZExt(b, v_ret, TypeStorageBool, "");
+
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b, LLVMIntEQ, v_ret,
+ l_sbool_const(1), ""),
+ opblocks[jumpaddr],
+ opblocks[opno + 1]);
+ break;
+ }
+ case EEOP_JSONEXPR_PATH:
build_EvalXFunc(b, mod, "ExecEvalJson",
v_state, op, v_econtext);
LLVMBuildBr(b, opblocks[opno + 1]);
break;
-
+ case EEOP_JSONEXPR_BEHAVIOR:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSONEXPR_COERCION:
+ /* XXX - handle! */
+ break;
+ case EEOP_JSON_ITEM_COERCION:
+ /* XXX - handle! */
+ break;
case EEOP_LAST:
Assert(false);
break;
diff --git a/src/backend/jit/llvm/llvmjit_types.c b/src/backend/jit/llvm/llvmjit_types.c
index b2bda86889..b30f20dd2d 100644
--- a/src/backend/jit/llvm/llvmjit_types.c
+++ b/src/backend/jit/llvm/llvmjit_types.c
@@ -133,6 +133,7 @@ void *referenced_functions[] =
ExecEvalXmlExpr,
ExecEvalJsonConstructor,
ExecEvalJsonIsPredicate,
+ ExecEvalJsonSkip,
ExecEvalJson,
MakeExpandedObjectReadOnlyInternal,
slot_getmissingattrs,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..9c02218155 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -901,7 +901,11 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
{
JsonExpr *jsexpr = (JsonExpr *) node;
- if (ExecEvalJsonNeedsSubTransaction(jsexpr, NULL))
+ /*
+ * XXX - don't really know why it makes sense to ignore the coercion
+ * part here.
+ */
+ if (ExecEvalJsonNeedsSubTransaction(jsexpr, false /* coerce */))
{
context->max_hazard = PROPARALLEL_UNSAFE;
return true;
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 1e3f1bbee8..4921589260 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -244,7 +244,11 @@ typedef enum ExprEvalOp
EEOP_SUBPLAN,
EEOP_JSON_CONSTRUCTOR,
EEOP_IS_JSON,
- EEOP_JSONEXPR,
+ EEOP_JSONEXPR_SKIP,
+ EEOP_JSONEXPR_PATH,
+ EEOP_JSONEXPR_BEHAVIOR,
+ EEOP_JSONEXPR_COERCION,
+ EEOP_JSON_ITEM_COERCION,
/* aggregation related nodes */
EEOP_AGG_STRICT_DESERIALIZE,
@@ -682,12 +686,44 @@ typedef struct ExprEvalStep
JsonIsPredicate *pred; /* original expression node */
} is_json;
- /* for EEOP_JSONEXPR */
+ /* for EEOP_JSONEXPR_PATH */
struct
{
struct JsonExprState *jsestate;
} jsonexpr;
+ /* for EEOP_JSONEXPR_SKIP */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_coercion;
+ } jsonexpr_skip;
+
+ /* for EEOP_JSONEXPR_BEHAVIOR */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_onerror_default;
+ int jump_onempty_default;
+ int jump_coercion;
+ int jump_skip_coercion;
+ } jsonexpr_behavior;
+
+ /* for EEOP_JSONEXPR_COERCION */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_coercion_error;
+ } jsonexpr_coercion;
+
+ struct
+ {
+ struct JsonItemCoercionsState *jcstate;
+ int jump_skip_item_coercion;
+ } json_item_coercion;
} d;
} ExprEvalStep;
@@ -747,49 +783,112 @@ typedef struct JsonConstructorExprState
int nargs;
} JsonConstructorExprState;
+/*
+ * State for EEOP_JSONEXPR_BEHAVIOR and EEOP_JSONEXPR_COERCION steps that gets
+ * filled in during EEOP_JSONEXPR_PATH.
+ */
+typedef struct JsonExprPostEvalState
+{
+ /* Is JSON item empty? */
+ bool empty;
+
+ /* Did JSON item evaluation cause an error? */
+ bool error;
+
+ /* Did coercion evaluation cause an error? */
+ bool coercion_error;
+
+ /* Has the result been coerced properly? */
+ bool coercion_done;
+
+ /* Use a sub-transaction when evaluating the coercion */
+ bool coercion_use_subtrans;
+
+ /* Cache for json_populate_type() called for coercion in some cases */
+ void *cache;
+
+ /* Coercion state; same as parent JsonExprState.coercions when not NULL */
+ ExprState *coercions;
+} JsonExprPostEvalState;
+
+/*
+ * Information computed before evaluating a JsonExpr expression.
+ */
+typedef struct JsonExprPreEvalState
+{
+ /* value/isnull for JsonExpr.formatted_expr */
+ NullableDatum formatted_expr;
+
+ /* value/isnull for JsonExpr.pathspec */
+ NullableDatum pathspec;
+
+ /* JsonPathVariableEvalContext entries for JsonExpr.passing_values */
+ List *args;
+} JsonExprPreEvalState;
+
/* EEOP_JSONEXPR state, too big to inline */
typedef struct JsonExprState
{
JsonExpr *jsexpr; /* original expression node */
+ JsonExprPreEvalState pre_eval;
+ JsonExprPostEvalState post_eval;
+
+ /*
+ * Should use a sub-transaction for path evaluation and subsequent
+ * coercion evaluation, if any?
+ */
+ bool needSubtrans;
+ bool coercionNeedSubtrans;
+
struct
{
- FmgrInfo func; /* typinput function for output type */
+ FmgrInfo *finfo; /* typinput function for output type */
Oid typioparam;
} input; /* I/O info for output type */
- NullableDatum
- *formatted_expr, /* formatted context item value */
- *res_expr, /* result item */
- *coercion_expr, /* input for JSON item coercion */
- *pathspec; /* path specification value */
+ /*
+ * Either of the following two is used by ExecEvalJsonExprCoercion() to
+ * apply coercion to the final result if needed.
+ */
+ ExprState *result_coercion;
+ ExprState *coercions;
+} JsonExprState;
- ExprState *result_expr; /* coerced to output type */
- ExprState *default_on_empty; /* ON EMPTY DEFAULT expression */
- ExprState *default_on_error; /* ON ERROR DEFAULT expression */
- List *args; /* passing arguments */
+/*
+ * State for a given member of JsonItemCoercions.
+ */
+typedef struct JsonItemCoercionState
+{
+ /* Expression used to evaluate the coercion */
+ JsonCoercion *coercion;
- void *cache; /* cache for json_populate_type() */
+ /* ExprEvalStep to compute this coercion's expression */
+ int jump_eval_expr;
+} JsonItemCoercionState;
- struct JsonCoercionsState
- {
- struct JsonCoercionState
- {
- JsonCoercion *coercion; /* coercion expression */
- ExprState *estate; /* coercion expression state */
- } null,
- string,
- numeric ,
- boolean,
- date,
- time,
- timetz,
- timestamp,
- timestamptz,
- composite;
- } coercions; /* states for coercion from SQL/JSON item
- * types directly to the output type */
-} JsonExprState;
+/*
+ * State for evaluating the coercion for a given JSON item using one of
+ * the following coercions.
+ *
+ * Note that while ExecInitExprRec() for JsonItemCoercions will initialize
+ * ExprEvalSteps for all of the members that need it, only one will get run
+ * during a given evaluation of the enclosing JsonExpr depending on the type
+ * of the result JSON item.
+ */
+typedef struct JsonItemCoercionsState
+{
+ JsonItemCoercionState null;
+ JsonItemCoercionState string;
+ JsonItemCoercionState numeric;
+ JsonItemCoercionState boolean;
+ JsonItemCoercionState date;
+ JsonItemCoercionState time;
+ JsonItemCoercionState timetz;
+ JsonItemCoercionState timestamp;
+ JsonItemCoercionState timestamptz;
+ JsonItemCoercionState composite;
+} JsonItemCoercionsState;
/* functions in execExpr.c */
extern void ExprEvalPushStep(ExprState *es, const ExprEvalStep *s);
@@ -850,14 +949,15 @@ extern void ExecEvalSysVar(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, TupleTableSlot *slot);
extern void ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
+extern bool ExecEvalJsonSkip(ExprEvalStep *op);
+extern int ExecEvalJsonExprBehavior(ExprEvalStep *op);
extern void ExecEvalJson(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
-extern Datum ExecPrepareJsonItemCoercion(struct JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pjcstate);
-extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *);
+JsonCoercion *ExecGetJsonItemCoercion(struct JsonbValue *item, JsonItemCoercions *coercions);
+extern Datum ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pjcstate);
+extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce);
extern Datum ExecEvalExprPassingCaseValue(ExprState *estate,
ExprContext *econtext, bool *isnull,
Datum caseval_datum,
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index bae466b523..6bdd9f5121 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -69,8 +69,10 @@ typedef enum
/* Convenience macros */
#define DatumGetJsonbP(d) ((Jsonb *) PG_DETOAST_DATUM(d))
+#define DatumGetJsonbValueP(d) ((JsonbValue *) DatumGetPointer(d))
#define DatumGetJsonbPCopy(d) ((Jsonb *) PG_DETOAST_DATUM_COPY(d))
#define JsonbPGetDatum(p) PointerGetDatum(p)
+#define JsonbValuePGetDatum(p) PointerGetDatum(p)
#define PG_GETARG_JSONB_P(x) DatumGetJsonbP(PG_GETARG_DATUM(x))
#define PG_GETARG_JSONB_P_COPY(x) DatumGetJsonbPCopy(PG_GETARG_DATUM(x))
#define PG_RETURN_JSONB_P(x) PG_RETURN_POINTER(x)
--
2.35.3
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-07-21 14:55 ` Amit Langote <amitlangote09@gmail.com>
2022-07-21 15:19 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 75+ messages in thread
From: Amit Langote @ 2022-07-21 14:55 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On Wed, Jul 20, 2022 at 11:09 PM Amit Langote <amitlangote09@gmail.com> wrote:
> On Wed, Jul 20, 2022 at 12:37 AM Andres Freund <andres@anarazel.de> wrote:
> > On 2022-07-19 20:40:11 +0900, Amit Langote wrote:
> > > About that, I was wondering if the blocks in llvm_compile_expr() need
> > > to be hand-coded to match what's added in ExecInterpExpr() or if I've
> > > missed some tool that can be used instead?
> >
> > The easiest way is to just call an external function for the implementation of
> > the step. But yes, otherwise you need to handcraft it.
>
> Ok, thanks.
>
> So I started updating llvm_compile_expr() for handling the new
> ExprEvalSteps that the patch adds to ExecExprInterp(), but quickly
> realized that code could have been consolidated into less code, or
> IOW, into fewer new ExprEvalSteps. So, I refactored things that way
> and am now retrying adding the code to llvm_compile_expr() based on
> new, better consolidated, code.
>
> Here's the updated version, without the llvm pieces, in case you'd
> like to look at it even in this state. I'll post a version with llvm
> pieces filled in tomorrow. (I have merged the different patches into
> one for convenience.)
And here's a version with llvm pieces filled in.
Because I wrote all of it while not really understanding how the LLVM
constructs like blocks and branches work, the only reason I think
those llvm_compile_expr() additions may be correct is that all the
tests in jsonb_sqljson.sql pass even if I add the following line at
the top:
set jit_above_cost to 0;
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
v4-0001-Some-improvements-to-JsonExpr-evaluation.patch (application/octet-stream, 51.7K) download
From d2d1164710518d83e44f5f89ad9c1e7d535b9667 Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Wed, 22 Jun 2022 12:00:47 -0400
Subject: [PATCH v4] Some improvements to JsonExpr evaluation
* Store a pointer to the returning type input function FmgrInfo,
not the struct itself, in JsonExprState
* Evaluate various JsonExpr sub-expressions using parent ExprState
These include PASSING args, ON ERROR, ON EMPTY default expressions.
PASSING args are simple, because they all must be evaluated before
the JSON path evaluation can occur anyway, so pushing those
expressions to be uncondionally evaluated before the step to
evaluate JsonExpr proper suffices. Evaluation of the ON ERROR and
the ON EMPTY expressions is conditional on what JSON item pops out
of the path evaluation (ExecEvalJson()), so needs more logic to
gate the steps for their expressions that are pushed into the parent
ExprState, rather than be imlemented using separate ExprStates.
To that end, this moves the ON ERROR, ON EMPTY behavior decision logic
out of ExecEvalJson() and ExecEvalJsonExpr() into a new function
ExecEvalJsonExprBehavior().
* Final result coercion is now also computed as a separate step that
gets pushed into the parent ExprState. Though given that any
coercion evaluation errors must be handled the same way as the errors
of JSON item evaluation, that is, to be handled according the ON ERROR
behavior specification, a sub-transaction must be used around coercion
evaluation, so the ExprState to use for the same must have to be a
different one from the JsonExpr's. However, instead of using one
ExprState for each JsonCoercion member of JsonItemCoercions, only one
is used for the whole JsonItemCoercions. The individual JsonCoercion
members (or their Exprs) are handled by steps pushed into that
ExprState, of which only the one that applies to a given JSON item is
chosen at the runtime and others "jumped" over.
---
src/backend/executor/execExpr.c | 381 +++++++++++++++++----
src/backend/executor/execExprInterp.c | 474 +++++++++++++++++++-------
src/backend/jit/llvm/llvmjit_expr.c | 137 +++++++-
src/backend/jit/llvm/llvmjit_types.c | 4 +
src/backend/optimizer/util/clauses.c | 6 +-
src/include/executor/execExpr.h | 206 ++++++++---
src/include/utils/jsonb.h | 2 +
7 files changed, 977 insertions(+), 233 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index c8d7145fe3..9e1f7b2b1a 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2558,113 +2558,360 @@ ExecInitExprRec(Expr *node, ExprState *state,
break;
}
+ case T_JsonCoercion:
+ {
+ JsonCoercion *coercion = castNode(JsonCoercion, node);
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
+
+ /*
+ * Only ever get here for JsonExpr.result_expression and the
+ * caller should have checked this.
+ */
+ Assert(coercion->expr);
+
+ /*
+ * First push a step to read the value provided by the parent
+ * JsonExpr via a CaseTestExpr.
+ */
+ scratch.opcode = EEOP_CASE_TESTVAL;
+ scratch.d.casetest.value = state->innermost_caseval;
+ scratch.d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, &scratch);
+
+ /* Push step(s) to compute coercion->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
+
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) coercion->expr, state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
+
+ break;
+ }
+
case T_JsonExpr:
{
JsonExpr *jexpr = castNode(JsonExpr, node);
JsonExprState *jsestate = palloc0(sizeof(JsonExprState));
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
ListCell *argexprlc;
ListCell *argnamelc;
+ int skip_step_off;
+ int passing_args_step_off;
+ int behavior_step_off;
+ int onempty_default_step_off;
+ int onerror_default_step_off;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
+ int coercion_step_off;
+ ExprEvalStep *as;
+ bool throwErrors =
+ (jexpr->on_error->btype == JSON_BEHAVIOR_ERROR);
+
+ jsestate->jsexpr = jexpr;
+
+ /*
+ * Add steps to compute formatted_expr, pathspec, and
+ * PASSING arg expressions as things that must be
+ * evaluated before the "main" JSON path evaluation.
+ */
+ ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
+ &pre_eval->formatted_expr.value,
+ &pre_eval->formatted_expr.isnull);
+ ExecInitExprRec((Expr *) jexpr->path_spec, state,
+ &pre_eval->pathspec.value,
+ &pre_eval->pathspec.isnull);
- scratch.opcode = EEOP_JSONEXPR;
+ /*
+ * However, before pushing steps for PASSING args, push a step
+ * that will decide whether to skip evaluating the args and
+ * JSON path evaluation depending on whether either of
+ * formatted_expr and pathspec is NULL.
+ */
+ scratch.opcode = EEOP_JSONEXPR_SKIP;
+ scratch.d.jsonexpr_skip.jsestate = jsestate;
+ skip_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ jsestate->pre_eval.args = NIL;
+ passing_args_step_off = state->steps_len;
+ forboth(argexprlc, jexpr->passing_values,
+ argnamelc, jexpr->passing_names)
+ {
+ Expr *argexpr = (Expr *) lfirst(argexprlc);
+ String *argname = lfirst_node(String, argnamelc);
+ JsonPathVariableEvalContext *var = palloc(sizeof(*var));
+
+ var->name = pstrdup(argname->sval);
+ var->typid = exprType((Node *) argexpr);
+ var->typmod = exprTypmod((Node *) argexpr);
+
+ /*
+ * Not necessary when being evaluated for a JsonExpr, like
+ * in this case.
+ */
+ var->estate = NULL;
+ var->econtext = NULL;
+ var->mcxt = NULL;
+
+ /*
+ * Mark these as always evaluated because they must have
+ * been evaluated before JSON path evaluation begins,
+ * because we haven't pushed the step for the latter yet.
+ */
+ var->evaluated = true;
+
+ ExecInitExprRec((Expr *) argexpr, state,
+ &var->value,
+ &var->isnull);
+
+ pre_eval->args = lappend(pre_eval->args, var);
+ }
+
+ /* Now push the step for the actual JSON path evaluation. */
+ scratch.opcode = EEOP_JSONEXPR_PATH;
scratch.d.jsonexpr.jsestate = jsestate;
+ ExprEvalPushStep(state, &scratch);
- jsestate->jsexpr = jexpr;
+ /*
+ * Now push steps to control the expressions evaluated based
+ * on the result of JSON path evaluation.
+ */
- jsestate->formatted_expr =
- palloc(sizeof(*jsestate->formatted_expr));
+ /*
+ * Step to handle ON ERROR and ON EMPTY behavior correctly.
+ */
+ scratch.opcode = EEOP_JSONEXPR_BEHAVIOR;
+ scratch.d.jsonexpr_behavior.jsestate = jsestate;
+ behavior_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
- ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
- &jsestate->formatted_expr->value,
- &jsestate->formatted_expr->isnull);
+ onempty_default_step_off = state->steps_len;
+ if (jexpr->on_empty)
+ {
+ /* Push step(s) for the default ON EMPTY expression. */
+ if (jexpr->on_empty->default_expr)
+ {
+ ExecInitExprRec((Expr *) jexpr->on_empty->default_expr,
+ state, resv, resnull);
- jsestate->pathspec =
- palloc(sizeof(*jsestate->pathspec));
+ /* Emit JUMP step to jump to end of JsonExpr code */
+ scratch.opcode = EEOP_JUMP;
+ scratch.d.jump.jumpdone = -1; /* computed later */
+ ExprEvalPushStep(state, &scratch);
- ExecInitExprRec((Expr *) jexpr->path_spec, state,
- &jsestate->pathspec->value,
- &jsestate->pathspec->isnull);
+ /*
+ * Don't know address for that jump yet, compute once the
+ * whole JsonExpr is built.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len - 1);
+ }
+ }
- jsestate->res_expr =
- palloc(sizeof(*jsestate->res_expr));
+ onerror_default_step_off = state->steps_len;
+ if (jexpr->on_error)
+ {
+ /* Push step(s) for the default ON ERROR expression. */
+ if (jexpr->on_error->default_expr)
+ {
+ ExecInitExprRec((Expr *) jexpr->on_error->default_expr,
+ state, resv, resnull);
- jsestate->result_expr = jexpr->result_coercion
- ? ExecInitExprWithCaseValue((Expr *) jexpr->result_coercion->expr,
- state->parent,
- &jsestate->res_expr->value,
- &jsestate->res_expr->isnull)
- : NULL;
+ /* Emit JUMP step to jump to end of JsonExpr code */
+ scratch.opcode = EEOP_JUMP;
+ scratch.d.jump.jumpdone = -1; /* computed later */
+ ExprEvalPushStep(state, &scratch);
- jsestate->default_on_empty = !jexpr->on_empty ? NULL :
- ExecInitExpr((Expr *) jexpr->on_empty->default_expr,
- state->parent);
+ /*
+ * Don't know address for that jump yet, compute once the
+ * whole JsonExpr is built.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len - 1);
+ }
+ }
+
+ /*
+ * Step to handle applying coercion correctly. Might be
+ * skipped if no coercion needs to be applied separately
+ * necessary to do so will be set later.
+ */
+ scratch.opcode = EEOP_JSONEXPR_COERCION;
+ scratch.d.jsonexpr_coercion.jsestate = jsestate;
+ coercion_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ /*
+ * Adjust jump target addresses in various post-eval steps now
+ * that we have all the steps in place.
+ */
+
+ /* EEOP_JSONEXPR_SKIP */
+ as = &state->steps[skip_step_off];
+ as->d.jsonexpr_skip.jump_coercion = coercion_step_off;
+ as->d.jsonexpr_skip.jump_passing_args = passing_args_step_off;
- jsestate->default_on_error =
- ExecInitExpr((Expr *) jexpr->on_error->default_expr,
- state->parent);
+ /* EEOP_JSONEXPR_BEHAVIOR */
+ as = &state->steps[behavior_step_off];
+ as->d.jsonexpr_behavior.jump_onerror_default = onerror_default_step_off;
+ as->d.jsonexpr_behavior.jump_onempty_default = onempty_default_step_off;
+ as->d.jsonexpr_behavior.jump_coercion = coercion_step_off;
+ as->d.jsonexpr_behavior.jump_skip_coercion = coercion_step_off + 1;
+ /* EEOP_JSONEXPR_COERCION */
+ as = &state->steps[coercion_step_off];
+ as->d.jsonexpr_coercion.jump_coercion_error = behavior_step_off;
+ as->d.jsonexpr_coercion.jump_coercion_done = coercion_step_off + 1;
+
+ foreach(lc, adjust_jumps)
+ {
+ as = &state->steps[lfirst_int(lc)];
+
+ as->d.jump.jumpdone = state->steps_len;
+ }
+
+ /*
+ * Set if we must use a sub-transaction around path evaluation
+ * that can be aborted on error instead of aborting the parent
+ * query.
+ */
+ jsestate->needSubtrans =
+ ExecEvalJsonNeedsSubTransaction(jexpr,
+ true /* coerce */);
+ jsestate->coercionNeedSubtrans =
+ (!jsestate->needSubtrans && !throwErrors);
+
+ /*
+ * Set RETURNING type's input function used by
+ * ExecEvalJsonExprCoercion().
+ */
if (jexpr->omit_quotes ||
(jexpr->result_coercion && jexpr->result_coercion->via_io))
{
Oid typinput;
+ FmgrInfo *finfo;
/* lookup the result type's input function */
getTypeInputInfo(jexpr->returning->typid, &typinput,
&jsestate->input.typioparam);
- fmgr_info(typinput, &jsestate->input.func);
+ finfo = palloc0(sizeof(FmgrInfo));
+ fmgr_info(typinput, finfo);
+ jsestate->input.finfo = finfo;
}
- jsestate->args = NIL;
+ if (jexpr->result_coercion && jexpr->result_coercion->expr)
+ jsestate->result_coercion =
+ ExecInitExprWithCaseValue((Expr *)
+ jexpr->result_coercion,
+ state->parent,
+ resv, resnull);
- forboth(argexprlc, jexpr->passing_values,
- argnamelc, jexpr->passing_names)
- {
- Expr *argexpr = (Expr *) lfirst(argexprlc);
- String *argname = lfirst_node(String, argnamelc);
- JsonPathVariableEvalContext *var = palloc(sizeof(*var));
+ /* Set up coercion related data structures. */
+ if (jexpr->coercions)
+ jsestate->coercions =
+ ExecInitExprWithCaseValue((Expr *) jexpr->coercions,
+ state->parent,
+ resv, resnull);
- var->name = pstrdup(argname->sval);
- var->typid = exprType((Node *) argexpr);
- var->typmod = exprTypmod((Node *) argexpr);
- var->estate = ExecInitExpr(argexpr, state->parent);
- var->econtext = NULL;
- var->mcxt = NULL;
- var->evaluated = false;
- var->value = (Datum) 0;
- var->isnull = true;
+ break;
+ }
- jsestate->args =
- lappend(jsestate->args, var);
- }
+ case T_JsonItemCoercions:
+ {
+ JsonItemCoercions *coercions = castNode(JsonItemCoercions,
+ node);
+ JsonItemCoercionsState *jcstate =
+ palloc0(sizeof(JsonItemCoercionsState));
+ JsonCoercion **coercion;
+ JsonItemCoercionState *cstate;
+ ExprEvalStep *as;
+ int json_item_coercion_step_id;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
- jsestate->cache = NULL;
+ /*
+ * First push a step to read the value provided by the parent
+ * JsonExpr via a CaseTestExpr.
+ */
+ scratch.opcode = EEOP_CASE_TESTVAL;
+ scratch.d.casetest.value = state->innermost_caseval;
+ scratch.d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, &scratch);
- if (jexpr->coercions)
+ /* Push the control step. */
+ scratch.opcode = EEOP_JSON_ITEM_COERCION;
+ scratch.d.jsonexpr_item_coercion.jcstate = jcstate;
+ json_item_coercion_step_id = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+ /* Will set jump_skip_coercion target address below. */
+
+ /*
+ * Now push the steps of individual coercion's expression, if
+ * needed.
+ */
+ for (cstate = &jcstate->null,
+ coercion = &coercions->null;
+ coercion <= &coercions->composite;
+ coercion++, cstate++)
{
- JsonCoercion **coercion;
- struct JsonCoercionState *cstate;
- Datum *caseval;
- bool *casenull;
+ cstate->coercion = *coercion;
+ if (cstate->coercion && cstate->coercion->expr)
+ {
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
- jsestate->coercion_expr =
- palloc(sizeof(*jsestate->coercion_expr));
+ cstate->jump_eval_expr = state->steps_len;
- caseval = &jsestate->coercion_expr->value;
- casenull = &jsestate->coercion_expr->isnull;
+ /* Push step(s) to compute (*coercion)->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
- for (cstate = &jsestate->coercions.null,
- coercion = &jexpr->coercions->null;
- coercion <= &jexpr->coercions->composite;
- coercion++, cstate++)
- {
- cstate->coercion = *coercion;
- cstate->estate = *coercion ?
- ExecInitExprWithCaseValue((Expr *) (*coercion)->expr,
- state->parent,
- caseval, casenull) : NULL;
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) cstate->coercion->expr,
+ state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
}
+ else
+ cstate->jump_eval_expr = -1;
+
+ /* Emit JUMP step to jump to end of coercions code */
+ scratch.opcode = EEOP_JUMP;
+
+ /*
+ * Remember JUMP step address to set the actual jump
+ * target address below.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len);
+ ExprEvalPushStep(state, &scratch);
+ }
+
+ /*
+ * Adjust the jump target address of the control step and all
+ * the jumps we added in the above loop to make them point to
+ * the step after the last step that would have been added
+ * above.
+ */
+ as = &state->steps[json_item_coercion_step_id];
+ as->d.jsonexpr_item_coercion.jump_skip_item_coercion = state->steps_len;
+ foreach(lc, adjust_jumps)
+ {
+ int jump_step_id = lfirst_int(lc);
+
+ as = &state->steps[jump_step_id];
+ as->d.jump.jumpdone = state->steps_len;
}
- ExprEvalPushStep(state, &scratch);
break;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 723770fda0..af24800d09 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -158,6 +158,15 @@ static TupleDesc get_cached_rowtype(Oid type_id, int32 typmod,
bool *changed);
static void ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, bool checkisnull);
+static Datum ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null);
+typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
+ Datum item, bool *resnull, void *p, bool *error);
+static Datum ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
+ ExprContext *econtext,
+ Datum res, bool *resnull,
+ void *p, bool *error, bool subtrans);
+static Datum ExecEvalJsonCoercion(ExprEvalStep *op, ExprContext *econtext,
+ Datum res, bool *isNull, void *p, bool *error);
/* fast-path evaluation functions */
static Datum ExecJustInnerVar(ExprState *state, ExprContext *econtext, bool *isnull);
@@ -490,7 +499,11 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SUBPLAN,
&&CASE_EEOP_JSON_CONSTRUCTOR,
&&CASE_EEOP_IS_JSON,
- &&CASE_EEOP_JSONEXPR,
+ &&CASE_EEOP_JSONEXPR_SKIP,
+ &&CASE_EEOP_JSONEXPR_PATH,
+ &&CASE_EEOP_JSONEXPR_BEHAVIOR,
+ &&CASE_EEOP_JSONEXPR_COERCION,
+ &&CASE_EEOP_JSON_ITEM_COERCION,
&&CASE_EEOP_AGG_STRICT_DESERIALIZE,
&&CASE_EEOP_AGG_DESERIALIZE,
&&CASE_EEOP_AGG_STRICT_INPUT_CHECK_ARGS,
@@ -1817,13 +1830,37 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
- EEO_CASE(EEOP_JSONEXPR)
+ EEO_CASE(EEOP_JSONEXPR_PATH)
{
/* too complex for an inline implementation */
ExecEvalJson(state, op, econtext);
EEO_NEXT();
}
+ EEO_CASE(EEOP_JSONEXPR_SKIP)
+ {
+ ExecEvalJsonExprSkip(state, op);
+ EEO_JUMP(op->d.jsonexpr_skip.jumpdone);
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_BEHAVIOR)
+ {
+ ExecEvalJsonExprBehavior(state, op);
+ EEO_JUMP(op->d.jsonexpr_behavior.jumpdone);
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_COERCION)
+ {
+ ExecEvalJsonExprCoercion(state, op, econtext);
+ EEO_JUMP(op->d.jsonexpr_coercion.jumpdone);
+ }
+
+ EEO_CASE(EEOP_JSON_ITEM_COERCION)
+ {
+ ExecEvalJsonExprItemCoercion(state, op);
+ EEO_JUMP(op->d.jsonexpr_item_coercion.jumpdone);
+ }
+
EEO_CASE(EEOP_LAST)
{
/* unreachable */
@@ -4602,8 +4639,7 @@ ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
* Evaluate a JSON error/empty behavior result.
*/
static Datum
-ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
- ExprState *default_estate, bool *is_null)
+ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null)
{
*is_null = false;
@@ -4628,7 +4664,9 @@ ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
return (Datum) 0;
case JSON_BEHAVIOR_DEFAULT:
- return ExecEvalExpr(default_estate, econtext, is_null);
+ /* Always handled in the caller. */
+ Assert(false);
+ return (Datum) 0;
default:
elog(ERROR, "unrecognized SQL/JSON behavior %d", behavior->btype);
@@ -4640,21 +4678,22 @@ ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
* Evaluate a coercion of a JSON item to the target type.
*/
static Datum
-ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
- Datum res, bool *isNull, void *p, bool *error)
+ExecEvalJsonCoercion(ExprEvalStep *op, ExprContext *econtext,
+ Datum res, bool *isNull, void *p, bool *error)
{
- ExprState *estate = p;
- JsonExprState *jsestate;
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ JsonExpr *jexpr = jsestate->jsexpr;
+ ExprState *estate = post_eval->coercions;
+
+ post_eval->coercion_error = false;
if (estate) /* coerce using specified expression */
return ExecEvalExpr(estate, econtext, isNull);
- jsestate = op->d.jsonexpr.jsestate;
-
- if (jsestate->jsexpr->op != JSON_EXISTS_OP)
+ if (jexpr->op != JSON_EXISTS_OP)
{
- JsonCoercion *coercion = jsestate->jsexpr->result_coercion;
- JsonExpr *jexpr = jsestate->jsexpr;
+ JsonCoercion *coercion = jexpr->result_coercion;
Jsonb *jb = *isNull ? NULL : DatumGetJsonbP(res);
if ((coercion && coercion->via_io) ||
@@ -4664,7 +4703,7 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
/* strip quotes and call typinput function */
char *str = *isNull ? NULL : JsonbUnquote(jb);
- return InputFunctionCall(&jsestate->input.func, str,
+ return InputFunctionCall(jsestate->input.finfo, str,
jsestate->input.typioparam,
jexpr->returning->typmod);
}
@@ -4672,17 +4711,20 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
return json_populate_type(res, JSONBOID,
jexpr->returning->typid,
jexpr->returning->typmod,
- &jsestate->cache,
+ &post_eval->cache,
econtext->ecxt_per_query_memory,
isNull);
}
- if (jsestate->result_expr)
+ /*
+ * Let the caller know that no coercion was done here, so it can
+ * coerce with jexpr->result_coercion if there's one.
+ */
+ if (jsestate->result_coercion)
{
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *isNull;
-
- res = ExecEvalExpr(jsestate->result_expr, econtext, isNull);
+ *op->resvalue = res;
+ *op->resnull = *isNull;
+ return ExecEvalExpr(jsestate->result_coercion, econtext, isNull);
}
return res;
@@ -4717,17 +4759,30 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
if (!var)
return -1;
- if (!var->evaluated)
+ /*
+ * When belonging to a JsonExpr, path variables are computed with the
+ * JsonExpr's ExprState (var->estate is NULL), so don't need to be computed
+ * here. In some other cases, such as when the path variables belonging
+ * to a JsonTable instead, those variables must be evaluated on their own,
+ * without the enclosing JsonExpr itself needing to be evaluated, so must
+ * be handled here.
+ */
+ if (var->estate && !var->evaluated)
{
MemoryContext oldcxt = var->mcxt ?
MemoryContextSwitchTo(var->mcxt) : NULL;
+ Assert(var->econtext != NULL);
var->value = ExecEvalExpr(var->estate, var->econtext, &var->isnull);
var->evaluated = true;
if (oldcxt)
MemoryContextSwitchTo(oldcxt);
}
+ else
+ {
+ Assert(var->evaluated);
+ }
if (var->isnull)
{
@@ -4741,19 +4796,84 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
return id;
}
+/*
+ * Return a coercion among those given in 'coercions' for given
+ * JSON item.
+ */
+JsonCoercion *
+ExecGetJsonItemCoercion(JsonbValue *item, JsonItemCoercions *coercions)
+{
+ if (item->type == jbvBinary &&
+ JsonContainerIsScalar(item->val.binary.data))
+ {
+ JsonbValue buf;
+ bool res PG_USED_FOR_ASSERTS_ONLY;
+
+ res = JsonbExtractScalar(item->val.binary.data, &buf);
+ item = &buf;
+ Assert(res);
+ }
+
+ /* get coercion state reference and datum of the corresponding SQL type */
+ switch (item->type)
+ {
+ case jbvNull:
+ return coercions->null;
+
+ case jbvString:
+ return coercions->string;
+
+ case jbvNumeric:
+ return coercions->numeric;
+
+ case jbvBool:
+ return coercions->boolean;
+
+ case jbvDatetime:
+ switch (item->val.datetime.typid)
+ {
+ case DATEOID:
+ return coercions->date;
+ case TIMEOID:
+ return coercions->time;
+ case TIMETZOID:
+ return coercions->timetz;
+ case TIMESTAMPOID:
+ return coercions->timestamp;
+ case TIMESTAMPTZOID:
+ return coercions->timestamptz;
+ default:
+ elog(ERROR, "unexpected jsonb datetime type oid %u",
+ item->val.datetime.typid);
+ }
+ break;
+
+ case jbvArray:
+ case jbvObject:
+ case jbvBinary:
+ return coercions->composite;
+
+ default:
+ elog(ERROR, "unexpected jsonb value type %d", item->type);
+ }
+
+ Assert(false);
+ return NULL;
+}
+
/*
* Prepare SQL/JSON item coercion to the output type. Returned a datum of the
* corresponding SQL type and a pointer to the coercion state.
*/
Datum
-ExecPrepareJsonItemCoercion(JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pcoercion)
+ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pcoercion)
{
- struct JsonCoercionState *coercion;
+ JsonbValue *item = DatumGetJsonbValueP(itemval);
+ JsonItemCoercionState *coercion;
Datum res;
- JsonbValue buf;
+ JsonbValue buf;
if (item->type == jbvBinary &&
JsonContainerIsScalar(item->val.binary.data))
@@ -4832,9 +4952,6 @@ ExecPrepareJsonItemCoercion(JsonbValue *item,
return res;
}
-typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
- Datum item, bool *resnull, void *p, bool *error);
-
static Datum
ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
ExprContext *econtext,
@@ -4905,7 +5022,6 @@ typedef struct
{
JsonPath *path;
bool *error;
- bool coercionInSubtrans;
} ExecEvalJsonExprContext;
static Datum
@@ -4916,16 +5032,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
ExecEvalJsonExprContext *cxt = pcxt;
JsonPath *path = cxt->path;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
- ExprState *estate = NULL;
- bool empty = false;
+ bool *empty = &post_eval->empty;
Datum res = (Datum) 0;
switch (jexpr->op)
{
case JSON_QUERY_OP:
- res = JsonPathQuery(item, path, jexpr->wrapper, &empty, error,
- jsestate->args);
+ res = JsonPathQuery(item, path, jexpr->wrapper, empty, error,
+ pre_eval->args);
if (error && *error)
{
*resnull = true;
@@ -4936,17 +5053,20 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
case JSON_VALUE_OP:
{
- struct JsonCoercionState *jcstate;
- JsonbValue *jbv = JsonPathValue(item, path, &empty, error,
- jsestate->args);
+ JsonCoercion *coercion;
+ JsonbValue *jbv = JsonPathValue(item, path, empty, error,
+ pre_eval->args);
if (error && *error)
+ {
+ *resnull = true;
return (Datum) 0;
+ }
if (!jbv) /* NULL or empty */
break;
- Assert(!empty);
+ Assert(!*empty);
*resnull = false;
@@ -4959,19 +5079,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
break;
}
- /* Use coercion from SQL/JSON item type to the output type */
- res = ExecPrepareJsonItemCoercion(jbv,
- jsestate->jsexpr->returning,
- &jsestate->coercions,
- &jcstate);
-
- if (jcstate->coercion &&
- (jcstate->coercion->via_io ||
- jcstate->coercion->via_populate))
+ /*
+ * Error out no cast exists to coerce SQL/JSON item to the
+ * the output type
+ */
+ coercion = ExecGetJsonItemCoercion(jbv, jsestate->jsexpr->coercions);
+ if (coercion && (coercion->via_io || coercion->via_populate))
{
if (error)
{
*error = true;
+ *resnull = true;
return (Datum) 0;
}
@@ -4983,32 +5101,30 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_SQL_JSON_ITEM_CANNOT_BE_CAST_TO_TARGET_TYPE),
errmsg("SQL/JSON item cannot be cast to target type")));
}
- else if (!jcstate->estate)
- return res; /* no coercion */
-
- /* coerce using specific expression */
- estate = jcstate->estate;
- jsestate->coercion_expr->value = res;
- jsestate->coercion_expr->isnull = *resnull;
+ else
+ {
+ /* Coerce using specific expression. */
+ res = JsonbValuePGetDatum(jbv);
+ post_eval->coercions = jsestate->coercions;
+ }
break;
}
case JSON_EXISTS_OP:
{
bool exists = JsonPathExists(item, path,
- jsestate->args,
+ pre_eval->args,
error);
*resnull = error && *error;
res = BoolGetDatum(exists);
- if (!jsestate->result_expr)
+ if (jexpr->result_coercion == NULL)
+ {
+ /* No coercion needed */
+ post_eval->coercion_done = true;
return res;
-
- /* coerce using result expression */
- estate = jsestate->result_expr;
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *resnull;
+ }
break;
}
@@ -5021,7 +5137,11 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
return (Datum) 0;
}
- if (empty)
+ /*
+ * If the ON EMPTY behavior is to cause an error, do so here. Other
+ * behaviors will be handled by the caller.
+ */
+ if (*empty)
{
Assert(jexpr->on_empty); /* it is not JSON_EXISTS */
@@ -5037,29 +5157,13 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_NO_SQL_JSON_ITEM),
errmsg("no SQL/JSON item")));
}
-
- if (jexpr->on_empty->btype == JSON_BEHAVIOR_DEFAULT)
-
- /*
- * Execute DEFAULT expression as a coercion expression, because
- * its result is already coerced to the target type.
- */
- estate = jsestate->default_on_empty;
- else
- /* Execute ON EMPTY behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_empty,
- jsestate->default_on_empty,
- resnull);
}
- return ExecEvalJsonExprSubtrans(ExecEvalJsonExprCoercion, op, econtext,
- res, resnull, estate, error,
- cxt->coercionInSubtrans);
+ return res;
}
bool
-ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *coercions)
+ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce)
{
if (jsexpr->on_error->btype == JSON_BEHAVIOR_ERROR)
return false;
@@ -5067,12 +5171,164 @@ ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
if (jsexpr->op == JSON_EXISTS_OP && !jsexpr->result_coercion)
return false;
- if (!coercions)
+ if (!coerce)
return true;
return false;
}
+/* Skip calling ExecEvalJson() on a JsonExpr? */
+void
+ExecEvalJsonExprSkip(ExprState *state, ExprEvalStep *op)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_skip.jsestate;
+ int *jump_to = &op->d.jsonexpr_skip.jumpdone;
+
+ /*
+ * Skip if either of the input expressions has turned out to be NULL,
+ * though do execute domain checks for NULLs, which are handled by the
+ * coercion step.
+ */
+ if (jsestate->pre_eval.formatted_expr.isnull ||
+ jsestate->pre_eval.pathspec.isnull)
+ {
+ *op->resvalue = 0;
+ *op->resnull = true;
+ Assert(jsestate->post_eval.coercions == NULL);
+ /* A signal to the coercion step to not use a sub-trancaction. */
+ jsestate->post_eval.coercion_error = true;
+ *jump_to = op->d.jsonexpr_skip.jump_coercion;
+ }
+ else
+ {
+ /*
+ * Go evaluate the PASSING args if any and subsequently JSON path
+ * itself.
+ */
+ *jump_to = op->d.jsonexpr_skip.jump_passing_args;
+ }
+}
+
+void
+ExecEvalJsonExprBehavior(ExprState *state, ExprEvalStep *op)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_behavior.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ JsonBehavior *behavior = NULL;
+ bool error = (post_eval->error || post_eval->coercion_error);
+ int *jump_to = &op->d.jsonexpr_behavior.jumpdone;
+
+ /*
+ * Set if the coercion step, if will run afterwards, should use a sub-
+ * transaction to be aborted on error instead of throwing the error.
+ * Don't use a sub-transaction if coercing the ON ERROR expression,
+ * though.
+ *
+ * XXX - doesn't that violate ON ERROR behavior? Actually, that is
+ * how it behaved even before this expression eval logic rewrite!
+ */
+ post_eval->coercion_use_subtrans =
+ (jsestate->coercionNeedSubtrans && !error);
+
+ /*
+ * If no error or the JSON item is not empty, directly go to the coercion
+ * step to coerce the item as is, or skip the coercion step if the item is
+ * already coerced by ExecEvalJson().
+ */
+ if (!error && !post_eval->empty)
+ {
+ *jump_to = !post_eval->coercion_done ?
+ op->d.jsonexpr_behavior.jump_coercion :
+ op->d.jsonexpr_behavior.jump_skip_coercion;
+ return;
+ }
+
+ if (error)
+ {
+ behavior = jsestate->jsexpr->on_error;
+ *jump_to = op->d.jsonexpr_behavior.jump_onerror_default;
+ }
+ else if (post_eval->empty)
+ {
+ behavior = jsestate->jsexpr->on_empty;
+ *jump_to = op->d.jsonexpr_behavior.jump_onempty_default;
+ }
+
+ Assert(behavior);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate value and go
+ * to the coercion step.
+ */
+ if (behavior->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ *op->resvalue = ExecEvalJsonBehavior(behavior, op->resnull);
+
+ if (post_eval->coercion_error)
+ {
+ post_eval->coercion_done = false;
+ post_eval->coercions = NULL;
+ }
+
+ *jump_to = op->d.jsonexpr_behavior.jump_coercion;
+ }
+
+ /*
+ * Else evaluate the default ON ERROR or ON EMPTY expression, with no
+ * coercion needed afterwards given that the expression is already
+ * coerced appropriately in the parser.
+ */
+
+ Assert(op->d.jsonexpr_behavior.jumpdone >= 0);
+}
+
+/*
+ * Apply coercion to a JSON item using either JsonExpr.result_coercion or one
+ * of JsonExpr.coercions, possibly using a sub-transaction.
+ */
+void
+ExecEvalJsonExprCoercion(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ int *jump_to = &op->d.jsonexpr_coercion.jumpdone;
+
+ *op->resvalue =
+ ExecEvalJsonExprSubtrans(ExecEvalJsonCoercion, op, econtext,
+ *op->resvalue, op->resnull, NULL,
+ &post_eval->coercion_error,
+ post_eval->coercion_use_subtrans);
+ if (post_eval->coercion_error)
+ *jump_to = op->d.jsonexpr_coercion.jump_coercion_error;
+ else
+ *jump_to = op->d.jsonexpr_coercion.jump_coercion_done;
+}
+
+/*
+ * Apply a coercion to a JSON item, choosing one among those present in
+ * JsonItemCoercions.
+ */
+void
+ExecEvalJsonExprItemCoercion(ExprState *state, ExprEvalStep *op)
+{
+ JsonItemCoercionsState *jcstate = op->d.jsonexpr_item_coercion.jcstate;
+ JsonItemCoercionState *cstate = NULL;
+ int *jump_to = &op->d.jsonexpr_item_coercion.jumpdone;
+
+ *op->resvalue =
+ ExecPrepareJsonItemCoercion(*op->resvalue, jcstate, &cstate);
+ if (cstate->coercion && cstate->coercion->expr)
+ {
+ Assert(cstate->jump_eval_expr >= 0);
+ *jump_to = cstate->jump_eval_expr;
+ }
+ else
+ {
+ /* Skip over all of the steps added for this JsonItemCoercions. */
+ *jump_to = op->d.jsonexpr_item_coercion.jump_skip_item_coercion;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEvalJson
* ----------------------------------------------------------------
@@ -5082,64 +5338,30 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
{
ExecEvalJsonExprContext cxt;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
Datum item;
Datum res = (Datum) 0;
JsonPath *path;
- ListCell *lc;
- bool error = false;
- bool needSubtrans;
bool throwErrors = jexpr->on_error->btype == JSON_BEHAVIOR_ERROR;
*op->resnull = true; /* until we get a result */
*op->resvalue = (Datum) 0;
- if (jsestate->formatted_expr->isnull || jsestate->pathspec->isnull)
- {
- /* execute domain checks for NULLs */
- (void) ExecEvalJsonExprCoercion(op, econtext, res, op->resnull,
- NULL, NULL);
-
- Assert(*op->resnull);
- return;
- }
-
- item = jsestate->formatted_expr->value;
- path = DatumGetJsonPathP(jsestate->pathspec->value);
+ item = pre_eval->formatted_expr.value;
+ path = DatumGetJsonPathP(pre_eval->pathspec.value);
- /* reset JSON path variable contexts */
- foreach(lc, jsestate->args)
- {
- JsonPathVariableEvalContext *var = lfirst(lc);
-
- var->econtext = econtext;
- var->evaluated = false;
- }
-
- needSubtrans = ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ /* Reset JsonExprPostEvalState for this evaluation. */
+ memset(post_eval, 0, sizeof(*post_eval));
cxt.path = path;
- cxt.error = throwErrors ? NULL : &error;
- cxt.coercionInSubtrans = !needSubtrans && !throwErrors;
- Assert(!needSubtrans || cxt.error);
+ cxt.error = throwErrors ? NULL : &post_eval->error;
+ Assert(!jsestate->needSubtrans || cxt.error);
res = ExecEvalJsonExprSubtrans(ExecEvalJsonExpr, op, econtext, item,
op->resnull, &cxt, cxt.error,
- needSubtrans);
-
- if (error)
- {
- /* Execute ON ERROR behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_error,
- jsestate->default_on_error,
- op->resnull);
-
- /* result is already coerced in DEFAULT behavior case */
- if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
- res = ExecEvalJsonExprCoercion(op, econtext, res,
- op->resnull,
- NULL, NULL);
- }
+ jsestate->needSubtrans);
*op->resvalue = res;
}
diff --git a/src/backend/jit/llvm/llvmjit_expr.c b/src/backend/jit/llvm/llvmjit_expr.c
index b6b6512ef1..443ac8761d 100644
--- a/src/backend/jit/llvm/llvmjit_expr.c
+++ b/src/backend/jit/llvm/llvmjit_expr.c
@@ -2359,12 +2359,147 @@ llvm_compile_expr(ExprState *state)
LLVMBuildBr(b, opblocks[opno + 1]);
break;
- case EEOP_JSONEXPR:
+ case EEOP_JSONEXPR_PATH:
build_EvalXFunc(b, mod, "ExecEvalJson",
v_state, op, v_econtext);
LLVMBuildBr(b, opblocks[opno + 1]);
break;
+ case EEOP_JSONEXPR_SKIP:
+ {
+ LLVMValueRef v_jumpaddrp;
+ LLVMValueRef v_jumpaddr;
+
+ build_EvalXFunc(b, mod, "ExecEvalJsonExprSkip",
+ v_state, op);
+ v_jumpaddrp = l_ptr_const(&op->d.jsonexpr_skip.jumpdone, l_ptr(TypeSizeT));
+ v_jumpaddr = LLVMBuildLoad(b, v_jumpaddrp, "");
+
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_skip.jump_coercion),
+ ""),
+ opblocks[op->d.jsonexpr_skip.jump_coercion],
+ opblocks[opno + 1]);
+ break;
+ }
+
+ case EEOP_JSONEXPR_BEHAVIOR:
+ {
+ LLVMValueRef v_jumpaddrp;
+ LLVMValueRef v_jumpaddr;
+ LLVMBasicBlockRef b_jump_skip_coercion,
+ b_jump_onerror_default;
+
+ build_EvalXFunc(b, mod, "ExecEvalJsonExprBehavior",
+ v_state, op);
+ v_jumpaddrp = l_ptr_const(&op->d.jsonexpr_behavior.jumpdone, l_ptr(TypeSizeT));
+ v_jumpaddr = LLVMBuildLoad(b, v_jumpaddrp, "");
+
+ b_jump_skip_coercion =
+ l_bb_before_v(opblocks[opno + 1],
+ "op.%d.jsonexpr_behavior_jump_skip_coercion", opno);
+ b_jump_onerror_default =
+ l_bb_before_v(opblocks[opno + 1],
+ "op.%d.jsonexpr_behavior_jump_onerror_default", opno);
+
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_behavior.jump_coercion),
+ ""),
+ opblocks[op->d.jsonexpr_behavior.jump_coercion],
+ b_jump_skip_coercion);
+
+ LLVMPositionBuilderAtEnd(b, b_jump_skip_coercion);
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_behavior.jump_skip_coercion),
+ ""),
+ opblocks[op->d.jsonexpr_behavior.jump_skip_coercion],
+ b_jump_onerror_default);
+
+ LLVMPositionBuilderAtEnd(b, b_jump_onerror_default);
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_behavior.jump_onerror_default),
+ ""),
+ opblocks[op->d.jsonexpr_behavior.jump_onerror_default],
+ opblocks[op->d.jsonexpr_behavior.jump_onempty_default]);
+ break;
+ }
+ case EEOP_JSONEXPR_COERCION:
+ {
+ LLVMValueRef v_jumpaddrp;
+ LLVMValueRef v_jumpaddr;
+
+ build_EvalXFunc(b, mod, "ExecEvalJsonExprCoercion",
+ v_state, op, v_econtext);
+ v_jumpaddrp = l_ptr_const(&op->d.jsonexpr_coercion.jumpdone, l_ptr(TypeSizeT));
+ v_jumpaddr = LLVMBuildLoad(b, v_jumpaddrp, "");
+
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_coercion.jump_coercion_error),
+ ""),
+ opblocks[op->d.jsonexpr_coercion.jump_coercion_error],
+ opblocks[op->d.jsonexpr_coercion.jump_coercion_done]);
+ break;
+ }
+ case EEOP_JSON_ITEM_COERCION:
+ {
+ JsonItemCoercionsState *jcstate = op->d.jsonexpr_item_coercion.jcstate;
+ JsonItemCoercionState *cstate;
+ LLVMValueRef v_jumpaddrp;
+ LLVMValueRef v_jumpaddr;
+ int n_coercions = (int) (&jcstate->composite - &jcstate->null) + 1;
+ int i;
+ LLVMBasicBlockRef *b_coercions;
+
+ build_EvalXFunc(b, mod, "ExecEvalJsonExprItemCoercion",
+ v_state, op);
+ v_jumpaddrp = l_ptr_const(&op->d.jsonexpr_item_coercion.jumpdone, l_ptr(TypeSizeT));
+ v_jumpaddr = LLVMBuildLoad(b, v_jumpaddrp, "");
+
+ b_coercions = palloc((n_coercions + 1) * sizeof(LLVMBasicBlockRef));
+ for (i = 0; i < n_coercions + 1; i++)
+ b_coercions[i] =
+ l_bb_before_v(opblocks[opno + 1],
+ "op.%d.json_item_coercion.%d",
+ opno, i);
+
+ LLVMBuildBr(b, b_coercions[0]);
+
+ /* Add conditional branches for individual coercion's expressions */
+ for (cstate = &jcstate->null, i = 0;
+ cstate <= &jcstate->composite;
+ cstate++, i++)
+ {
+ LLVMPositionBuilderAtEnd(b, b_coercions[i]);
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(cstate->jump_eval_expr),
+ ""),
+ cstate->jump_eval_expr >= 0 ?
+ opblocks[cstate->jump_eval_expr] :
+ opblocks[op->d.jsonexpr_item_coercion.jump_skip_item_coercion],
+ b_coercions[i + 1]);
+ }
+ LLVMPositionBuilderAtEnd(b, b_coercions[i]);
+ LLVMBuildBr(b, opblocks[op->d.jsonexpr_item_coercion.jump_skip_item_coercion]);
+ break;
+ }
case EEOP_LAST:
Assert(false);
break;
diff --git a/src/backend/jit/llvm/llvmjit_types.c b/src/backend/jit/llvm/llvmjit_types.c
index b2bda86889..34e13b4a7e 100644
--- a/src/backend/jit/llvm/llvmjit_types.c
+++ b/src/backend/jit/llvm/llvmjit_types.c
@@ -133,6 +133,10 @@ void *referenced_functions[] =
ExecEvalXmlExpr,
ExecEvalJsonConstructor,
ExecEvalJsonIsPredicate,
+ ExecEvalJsonExprSkip,
+ ExecEvalJsonExprBehavior,
+ ExecEvalJsonExprCoercion,
+ ExecEvalJsonExprItemCoercion,
ExecEvalJson,
MakeExpandedObjectReadOnlyInternal,
slot_getmissingattrs,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..9c02218155 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -901,7 +901,11 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
{
JsonExpr *jsexpr = (JsonExpr *) node;
- if (ExecEvalJsonNeedsSubTransaction(jsexpr, NULL))
+ /*
+ * XXX - don't really know why it makes sense to ignore the coercion
+ * part here.
+ */
+ if (ExecEvalJsonNeedsSubTransaction(jsexpr, false /* coerce */))
{
context->max_hazard = PROPARALLEL_UNSAFE;
return true;
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 1e3f1bbee8..a40ffd78d0 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -244,7 +244,11 @@ typedef enum ExprEvalOp
EEOP_SUBPLAN,
EEOP_JSON_CONSTRUCTOR,
EEOP_IS_JSON,
- EEOP_JSONEXPR,
+ EEOP_JSONEXPR_SKIP,
+ EEOP_JSONEXPR_PATH,
+ EEOP_JSONEXPR_BEHAVIOR,
+ EEOP_JSONEXPR_COERCION,
+ EEOP_JSON_ITEM_COERCION,
/* aggregation related nodes */
EEOP_AGG_STRICT_DESERIALIZE,
@@ -682,12 +686,71 @@ typedef struct ExprEvalStep
JsonIsPredicate *pred; /* original expression node */
} is_json;
- /* for EEOP_JSONEXPR */
+ /* for EEOP_JSONEXPR_PATH */
struct
{
struct JsonExprState *jsestate;
} jsonexpr;
+ /* for EEOP_JSONEXPR_SKIP */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_coercion;
+ int jump_passing_args;
+
+ /*
+ * ExecEvalJsonExprSkip() sets this to one of the above
+ * addresses.
+ */
+ int jumpdone;
+ } jsonexpr_skip;
+
+ /* for EEOP_JSONEXPR_BEHAVIOR */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_onerror_default;
+ int jump_onempty_default;
+ int jump_coercion;
+ int jump_skip_coercion;
+
+ /*
+ * ExecEvalJsonExprBehavior() sets this to one of the above
+ * addresses.
+ */
+ int jumpdone;
+ } jsonexpr_behavior;
+
+ /* for EEOP_JSONEXPR_COERCION */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_coercion_error;
+ int jump_coercion_done;
+
+ /*
+ * ExecEvalJsonExprCoercion() sets this to one of the above
+ * addresses.
+ */
+ int jumpdone;
+ } jsonexpr_coercion;
+
+ struct
+ {
+ struct JsonItemCoercionsState *jcstate;
+ int jump_skip_item_coercion;
+
+ /*
+ * ExecEvalJsonExprItemCoercion() sets this to either the "skip"
+ * address above or to the coercion expression's address given in
+ * one of the JsonItemCoercionStates found in 'jcstate'.
+ */
+ int jumpdone;
+ } jsonexpr_item_coercion;
} d;
} ExprEvalStep;
@@ -747,49 +810,112 @@ typedef struct JsonConstructorExprState
int nargs;
} JsonConstructorExprState;
+/*
+ * State for EEOP_JSONEXPR_BEHAVIOR and EEOP_JSONEXPR_COERCION steps that gets
+ * filled in during EEOP_JSONEXPR_PATH.
+ */
+typedef struct JsonExprPostEvalState
+{
+ /* Is JSON item empty? */
+ bool empty;
+
+ /* Did JSON item evaluation cause an error? */
+ bool error;
+
+ /* Did coercion evaluation cause an error? */
+ bool coercion_error;
+
+ /* Has the result been coerced properly? */
+ bool coercion_done;
+
+ /* Use a sub-transaction when evaluating the coercion */
+ bool coercion_use_subtrans;
+
+ /* Cache for json_populate_type() called for coercion in some cases */
+ void *cache;
+
+ /* Coercion state; same as parent JsonExprState.coercions when not NULL */
+ ExprState *coercions;
+} JsonExprPostEvalState;
+
+/*
+ * Information computed before evaluating a JsonExpr expression.
+ */
+typedef struct JsonExprPreEvalState
+{
+ /* value/isnull for JsonExpr.formatted_expr */
+ NullableDatum formatted_expr;
+
+ /* value/isnull for JsonExpr.pathspec */
+ NullableDatum pathspec;
+
+ /* JsonPathVariableEvalContext entries for JsonExpr.passing_values */
+ List *args;
+} JsonExprPreEvalState;
+
/* EEOP_JSONEXPR state, too big to inline */
typedef struct JsonExprState
{
JsonExpr *jsexpr; /* original expression node */
+ JsonExprPreEvalState pre_eval;
+ JsonExprPostEvalState post_eval;
+
+ /*
+ * Should use a sub-transaction for path evaluation and subsequent
+ * coercion evaluation, if any?
+ */
+ bool needSubtrans;
+ bool coercionNeedSubtrans;
+
struct
{
- FmgrInfo func; /* typinput function for output type */
+ FmgrInfo *finfo; /* typinput function for output type */
Oid typioparam;
} input; /* I/O info for output type */
- NullableDatum
- *formatted_expr, /* formatted context item value */
- *res_expr, /* result item */
- *coercion_expr, /* input for JSON item coercion */
- *pathspec; /* path specification value */
+ /*
+ * Either of the following two is used by ExecEvalJsonExprCoercion() to
+ * apply coercion to the final result if needed.
+ */
+ ExprState *result_coercion;
+ ExprState *coercions;
+} JsonExprState;
- ExprState *result_expr; /* coerced to output type */
- ExprState *default_on_empty; /* ON EMPTY DEFAULT expression */
- ExprState *default_on_error; /* ON ERROR DEFAULT expression */
- List *args; /* passing arguments */
+/*
+ * State for a given member of JsonItemCoercions.
+ */
+typedef struct JsonItemCoercionState
+{
+ /* Expression used to evaluate the coercion */
+ JsonCoercion *coercion;
- void *cache; /* cache for json_populate_type() */
+ /* ExprEvalStep to compute this coercion's expression */
+ int jump_eval_expr;
+} JsonItemCoercionState;
- struct JsonCoercionsState
- {
- struct JsonCoercionState
- {
- JsonCoercion *coercion; /* coercion expression */
- ExprState *estate; /* coercion expression state */
- } null,
- string,
- numeric ,
- boolean,
- date,
- time,
- timetz,
- timestamp,
- timestamptz,
- composite;
- } coercions; /* states for coercion from SQL/JSON item
- * types directly to the output type */
-} JsonExprState;
+/*
+ * State for evaluating the coercion for a given JSON item using one of
+ * the following coercions.
+ *
+ * Note that while ExecInitExprRec() for JsonItemCoercions will initialize
+ * ExprEvalSteps for all of the members that need it, only one will get run
+ * during a given evaluation of the enclosing JsonExpr depending on the type
+ * of the result JSON item.
+ */
+typedef struct JsonItemCoercionsState
+{
+ JsonItemCoercionState null;
+ JsonItemCoercionState string;
+ JsonItemCoercionState numeric;
+ JsonItemCoercionState boolean;
+ JsonItemCoercionState date;
+ JsonItemCoercionState time;
+ JsonItemCoercionState timetz;
+ JsonItemCoercionState timestamp;
+ JsonItemCoercionState timestamptz;
+ JsonItemCoercionState composite;
+} JsonItemCoercionsState;
/* functions in execExpr.c */
extern void ExprEvalPushStep(ExprState *es, const ExprEvalStep *s);
@@ -850,14 +976,18 @@ extern void ExecEvalSysVar(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, TupleTableSlot *slot);
extern void ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
+extern void ExecEvalJsonExprSkip(ExprState *state, ExprEvalStep *op);
+extern void ExecEvalJsonExprBehavior(ExprState *state, ExprEvalStep *op);
+extern void ExecEvalJsonExprCoercion(ExprState *state, ExprEvalStep *op,
+ ExprContext *econtext);
+extern void ExecEvalJsonExprItemCoercion(ExprState *state, ExprEvalStep *op);
extern void ExecEvalJson(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
-extern Datum ExecPrepareJsonItemCoercion(struct JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pjcstate);
-extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *);
+JsonCoercion *ExecGetJsonItemCoercion(struct JsonbValue *item, JsonItemCoercions *coercions);
+extern Datum ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pjcstate);
+extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce);
extern Datum ExecEvalExprPassingCaseValue(ExprState *estate,
ExprContext *econtext, bool *isnull,
Datum caseval_datum,
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index bae466b523..6bdd9f5121 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -69,8 +69,10 @@ typedef enum
/* Convenience macros */
#define DatumGetJsonbP(d) ((Jsonb *) PG_DETOAST_DATUM(d))
+#define DatumGetJsonbValueP(d) ((JsonbValue *) DatumGetPointer(d))
#define DatumGetJsonbPCopy(d) ((Jsonb *) PG_DETOAST_DATUM_COPY(d))
#define JsonbPGetDatum(p) PointerGetDatum(p)
+#define JsonbValuePGetDatum(p) PointerGetDatum(p)
#define PG_GETARG_JSONB_P(x) DatumGetJsonbP(PG_GETARG_DATUM(x))
#define PG_GETARG_JSONB_P_COPY(x) DatumGetJsonbPCopy(PG_GETARG_DATUM(x))
#define PG_RETURN_JSONB_P(x) PG_RETURN_POINTER(x)
--
2.35.3
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-07-21 15:19 ` Amit Langote <amitlangote09@gmail.com>
1 sibling, 0 replies; 75+ messages in thread
From: Amit Langote @ 2022-07-21 15:19 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Michael Paquier <michael@paquier.xyz>; Nikita Glukhov <n.gluhov@postgrespro.ru>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Tom Lane <tgl@sss.pgh.pa.us>
On Thu, Jul 21, 2022 at 11:55 PM Amit Langote <amitlangote09@gmail.com>
wrote:
> On Wed, Jul 20, 2022 at 11:09 PM Amit Langote <amitlangote09@gmail.com>
wrote:
> > On Wed, Jul 20, 2022 at 12:37 AM Andres Freund <andres@anarazel.de>
wrote:
> > > On 2022-07-19 20:40:11 +0900, Amit Langote wrote:
> > > > About that, I was wondering if the blocks in llvm_compile_expr()
need
> > > > to be hand-coded to match what's added in ExecInterpExpr() or if
I've
> > > > missed some tool that can be used instead?
> > >
> > > The easiest way is to just call an external function for the
implementation of
> > > the step. But yes, otherwise you need to handcraft it.
> >
> > Ok, thanks.
> >
> > So I started updating llvm_compile_expr() for handling the new
> > ExprEvalSteps that the patch adds to ExecExprInterp(), but quickly
> > realized that code could have been consolidated into less code, or
> > IOW, into fewer new ExprEvalSteps. So, I refactored things that way
> > and am now retrying adding the code to llvm_compile_expr() based on
> > new, better consolidated, code.
> >
> > Here's the updated version, without the llvm pieces, in case you'd
> > like to look at it even in this state. I'll post a version with llvm
> > pieces filled in tomorrow. (I have merged the different patches into
> > one for convenience.)
>
> And here's a version with llvm pieces filled in.
>
> Because I wrote all of it while not really understanding how the LLVM
> constructs like blocks and branches work, the only reason I think
> those llvm_compile_expr() additions may be correct is that all the
> tests in jsonb_sqljson.sql pass even if I add the following line at
> the top:
>
> set jit_above_cost to 0;
Oh and I did build --with-llvm. :-)
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-07-21 17:12 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
1 sibling, 1 reply; 75+ messages in thread
From: Alvaro Herrera @ 2022-07-21 17:12 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-Jul-21, Amit Langote wrote:
> Because I wrote all of it while not really understanding how the LLVM
> constructs like blocks and branches work, the only reason I think
> those llvm_compile_expr() additions may be correct is that all the
> tests in jsonb_sqljson.sql pass even if I add the following line at
> the top:
I suggest to build with --enable-coverage, then run the regression tests
and do "make coverage-html" and see if your code appears covered in the
report.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"¿Cómo puedes confiar en algo que pagas y que no ves,
y no confiar en algo que te dan y te lo muestran?" (Germán Poo)
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2022-07-22 03:21 ` Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Amit Langote @ 2022-07-22 03:21 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Andres Freund <andres@anarazel.de>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On Fri, Jul 22, 2022 at 2:12 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2022-Jul-21, Amit Langote wrote:
>
> > Because I wrote all of it while not really understanding how the LLVM
> > constructs like blocks and branches work, the only reason I think
> > those llvm_compile_expr() additions may be correct is that all the
> > tests in jsonb_sqljson.sql pass even if I add the following line at
> > the top:
>
> I suggest to build with --enable-coverage, then run the regression tests
> and do "make coverage-html" and see if your code appears covered in the
> report.
Thanks for the suggestion. I just did and it seems that both the
additions to ExecInterpExpr() and to llvm_compile_expr() are well
covered.
BTW, the only way I found to *forcefully* exercise llvm_compile_expr()
is to add `set jit_above_cost to 0` at the top of the test file, or
are we missing a force_jit_mode, like there is force_parallel_mode?
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-07-22 04:13 ` David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: David Rowley @ 2022-07-22 04:13 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Andres Freund <andres@anarazel.de>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On Fri, 22 Jul 2022 at 15:22, Amit Langote <amitlangote09@gmail.com> wrote:
> BTW, the only way I found to *forcefully* exercise llvm_compile_expr()
> is to add `set jit_above_cost to 0` at the top of the test file, or
> are we missing a force_jit_mode, like there is force_parallel_mode?
I don't think we'd need any setting to hide the JIT counters from
EXPLAIN ANALYZE since those only show with COSTS ON, which we tend not
to do.
I think for testing, you could just zero all the jit*above_cost GUCs.
If you look at the config_extra in [1], you'll see that animal runs
the tests with modified JIT parameters.
BTW, I was working on code inside llvm_compile_expr() a few days ago
and I thought I'd gotten the new evaluation steps I was adding correct
as it worked fine with jit_above_cost=0, but on further testing, it
crashed with jit_inline_above_cost=0. Might be worth doing both to see
if everything works as intended.
David
[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=desmoxytes&dt=2022-07-22%2003%3A04%3A03
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
@ 2022-07-22 05:49 ` Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Amit Langote @ 2022-07-22 05:49 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Andres Freund <andres@anarazel.de>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On Fri, Jul 22, 2022 at 1:13 PM David Rowley <dgrowleyml@gmail.com> wrote:
> On Fri, 22 Jul 2022 at 15:22, Amit Langote <amitlangote09@gmail.com> wrote:
> > BTW, the only way I found to *forcefully* exercise llvm_compile_expr()
> > is to add `set jit_above_cost to 0` at the top of the test file, or
> > are we missing a force_jit_mode, like there is force_parallel_mode?
>
> I don't think we'd need any setting to hide the JIT counters from
> EXPLAIN ANALYZE since those only show with COSTS ON, which we tend not
> to do.
Ah, makes sense.
> I think for testing, you could just zero all the jit*above_cost GUCs.
>
> If you look at the config_extra in [1], you'll see that animal runs
> the tests with modified JIT parameters.
>
> BTW, I was working on code inside llvm_compile_expr() a few days ago
> and I thought I'd gotten the new evaluation steps I was adding correct
> as it worked fine with jit_above_cost=0, but on further testing, it
> crashed with jit_inline_above_cost=0. Might be worth doing both to see
> if everything works as intended.
Thanks for the pointer.
So I didn't see things going bust on re-testing with all
jit_*_above_cost parameters set to 0, so maybe the
llvm_compile_expression() additions are alright.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-07-27 08:01 ` Amit Langote <amitlangote09@gmail.com>
2022-07-29 18:27 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 75+ messages in thread
From: Amit Langote @ 2022-07-27 08:01 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Andres Freund <andres@anarazel.de>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On Fri, Jul 22, 2022 at 2:49 PM Amit Langote <amitlangote09@gmail.com> wrote:
> On Fri, Jul 22, 2022 at 1:13 PM David Rowley <dgrowleyml@gmail.com> wrote:
> > BTW, I was working on code inside llvm_compile_expr() a few days ago
> > and I thought I'd gotten the new evaluation steps I was adding correct
> > as it worked fine with jit_above_cost=0, but on further testing, it
> > crashed with jit_inline_above_cost=0. Might be worth doing both to see
> > if everything works as intended.
>
> Thanks for the pointer.
>
> So I didn't see things going bust on re-testing with all
> jit_*_above_cost parameters set to 0, so maybe the
> llvm_compile_expression() additions are alright.
Here's an updated version of the patch, with mostly cosmetic changes.
In particular, I added comments describing the new llvm_compile_expr()
blobs.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
v5-0001-Some-improvements-to-JsonExpr-evaluation.patch (application/octet-stream, 55.7K) download
From 5e119f3bc16c3755709dd8ab9d0a3c9d4bb81319 Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Wed, 22 Jun 2022 12:00:47 -0400
Subject: [PATCH v5] Some improvements to JsonExpr evaluation
* Store a pointer to the returning type input function FmgrInfo,
not the struct itself, in JsonExprState
* Evaluate various JsonExpr sub-expressions using parent ExprState
These include PASSING args, ON ERROR, ON EMPTY default expressions.
PASSING args are simple, because they all must be evaluated before
the JSON path evaluation can occur anyway, so pushing those
expressions to be uncondionally evaluated before the step to
evaluate JsonExpr proper suffices. Evaluation of the ON ERROR and
the ON EMPTY expressions is conditional on what JSON item pops out
of the path evaluation (ExecEvalJson()), so needs more logic to
gate the steps for their expressions that are pushed into the parent
ExprState, rather than be imlemented using separate ExprStates.
To that end, this moves the ON ERROR, ON EMPTY behavior decision logic
out of ExecEvalJson() and ExecEvalJsonExpr() into a new function
ExecEvalJsonExprBehavior().
* Final result coercion is now also computed as a separate step that
gets pushed into the parent ExprState. Though given that any
coercion evaluation errors must be handled the same way as the errors
of JSON item evaluation, that is, to be handled according the ON ERROR
behavior specification, a sub-transaction must be used around coercion
evaluation, so the ExprState to use for the same must have to be a
different one from the JsonExpr's. However, instead of using one
ExprState for each JsonCoercion member of JsonItemCoercions, only one
is used for the whole JsonItemCoercions. The individual JsonCoercion
members (or their Exprs) are handled by steps pushed into that
ExprState, of which only the one that applies to a given JSON item is
chosen at the runtime and others "jumped" over.
* Move the logic of deciding whether to use a sub-transaction for JSON
path evaluation and subsequent coercion to ExecInitExprRec() instead
of recomputing the same thing on every evaluation.
---
src/backend/executor/execExpr.c | 385 +++++++++++++++++----
src/backend/executor/execExprInterp.c | 477 +++++++++++++++++++-------
src/backend/jit/llvm/llvmjit_expr.c | 219 +++++++++++-
src/backend/jit/llvm/llvmjit_types.c | 4 +
src/backend/optimizer/util/clauses.c | 6 +-
src/include/executor/execExpr.h | 211 ++++++++++--
src/include/utils/jsonb.h | 2 +
7 files changed, 1073 insertions(+), 231 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index c8d7145fe3..d4b5102707 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -178,7 +178,7 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
* ExecInitExprWithCaseValue: prepare an expression tree for execution
*
* This is the same as ExecInitExpr, except that a pointer to the value for
- * CasTestExpr is passed here.
+ * CaseTestExpr is passed here.
*/
ExprState *
ExecInitExprWithCaseValue(Expr *node, PlanState *parent,
@@ -2562,109 +2562,358 @@ ExecInitExprRec(Expr *node, ExprState *state,
{
JsonExpr *jexpr = castNode(JsonExpr, node);
JsonExprState *jsestate = palloc0(sizeof(JsonExprState));
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
ListCell *argexprlc;
ListCell *argnamelc;
+ int skip_step_off;
+ int passing_args_step_off;
+ int behavior_step_off;
+ int onempty_default_step_off;
+ int onerror_default_step_off;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
+ int coercion_step_off;
+ ExprEvalStep *as;
+ bool throwErrors =
+ (jexpr->on_error->btype == JSON_BEHAVIOR_ERROR);
+
+ jsestate->jsexpr = jexpr;
+
+ /*
+ * Add steps to compute formatted_expr, pathspec, and
+ * PASSING arg expressions as things that must be
+ * evaluated before the actual JSON path expression.
+ */
+ ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
+ &pre_eval->formatted_expr.value,
+ &pre_eval->formatted_expr.isnull);
+ ExecInitExprRec((Expr *) jexpr->path_spec, state,
+ &pre_eval->pathspec.value,
+ &pre_eval->pathspec.isnull);
+
+ /*
+ * Before pushing steps for PASSING args, push a step to
+ * decide whether to skip evaluating the args and the JSON
+ * path expression depending on whether either of
+ * formatted_expr and pathspec is NULL.
+ */
+ scratch.opcode = EEOP_JSONEXPR_SKIP;
+ scratch.d.jsonexpr_skip.jsestate = jsestate;
+ skip_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ /* PASSING args. */
+ jsestate->pre_eval.args = NIL;
+ passing_args_step_off = state->steps_len;
+ forboth(argexprlc, jexpr->passing_values,
+ argnamelc, jexpr->passing_names)
+ {
+ Expr *argexpr = (Expr *) lfirst(argexprlc);
+ String *argname = lfirst_node(String, argnamelc);
+ JsonPathVariableEvalContext *var = palloc(sizeof(*var));
- scratch.opcode = EEOP_JSONEXPR;
+ var->name = pstrdup(argname->sval);
+ var->typid = exprType((Node *) argexpr);
+ var->typmod = exprTypmod((Node *) argexpr);
+
+ /*
+ * Not necessary when being evaluated for a JsonExpr, like
+ * in this case.
+ */
+ var->estate = NULL;
+ var->econtext = NULL;
+ var->mcxt = NULL;
+
+ /*
+ * Mark these as always evaluated because they must have
+ * been evaluated before JSON path evaluation begins,
+ * because we haven't pushed the step for the latter yet.
+ */
+ var->evaluated = true;
+
+ ExecInitExprRec((Expr *) argexpr, state,
+ &var->value,
+ &var->isnull);
+
+ pre_eval->args = lappend(pre_eval->args, var);
+ }
+
+ /* Step for the actual JSON path evaluation. */
+ scratch.opcode = EEOP_JSONEXPR_PATH;
scratch.d.jsonexpr.jsestate = jsestate;
+ ExprEvalPushStep(state, &scratch);
- jsestate->jsexpr = jexpr;
+ /*
+ * Push steps to control the evaluation of expressions based
+ * on the result of JSON path evaluation.
+ */
- jsestate->formatted_expr =
- palloc(sizeof(*jsestate->formatted_expr));
+ /* Step to handle ON ERROR and ON EMPTY behavior */
+ scratch.opcode = EEOP_JSONEXPR_BEHAVIOR;
+ scratch.d.jsonexpr_behavior.jsestate = jsestate;
+ behavior_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
- ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
- &jsestate->formatted_expr->value,
- &jsestate->formatted_expr->isnull);
+ /* Step(s) to evaluate ON EMPTY default expression */
+ onempty_default_step_off = state->steps_len;
+ if (jexpr->on_empty && jexpr->on_empty->default_expr)
+ {
+ ExecInitExprRec((Expr *) jexpr->on_empty->default_expr,
+ state, resv, resnull);
- jsestate->pathspec =
- palloc(sizeof(*jsestate->pathspec));
+ /*
+ * Emit JUMP step to jump to end of JsonExpr code, because
+ * evaluating the default expression gives the final
+ * result and there's nothing more to do.
+ */
+ scratch.opcode = EEOP_JUMP;
+ scratch.d.jump.jumpdone = -1; /* computed later */
+ ExprEvalPushStep(state, &scratch);
- ExecInitExprRec((Expr *) jexpr->path_spec, state,
- &jsestate->pathspec->value,
- &jsestate->pathspec->isnull);
+ /*
+ * Don't know address for that jump yet, compute once the
+ * whole JsonExpr is built.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len - 1);
+ }
- jsestate->res_expr =
- palloc(sizeof(*jsestate->res_expr));
+ /* Step(s) to evaluate ON ERROR default expression */
+ onerror_default_step_off = state->steps_len;
+ if (jexpr->on_error && jexpr->on_error->default_expr)
+ {
+ ExecInitExprRec((Expr *) jexpr->on_error->default_expr,
+ state, resv, resnull);
- jsestate->result_expr = jexpr->result_coercion
- ? ExecInitExprWithCaseValue((Expr *) jexpr->result_coercion->expr,
- state->parent,
- &jsestate->res_expr->value,
- &jsestate->res_expr->isnull)
- : NULL;
+ /* See the comment above. */
+ scratch.opcode = EEOP_JUMP;
+ scratch.d.jump.jumpdone = -1; /* computed later */
+ ExprEvalPushStep(state, &scratch);
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len - 1);
+ }
+
+ /* Step to handle applying coercion */
+ scratch.opcode = EEOP_JSONEXPR_COERCION;
+ scratch.d.jsonexpr_coercion.jsestate = jsestate;
+ coercion_step_off = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+
+ /*
+ * Adjust jump target addresses in various post-eval steps now
+ * that we have all the steps in place.
+ */
+
+ /* EEOP_JSONEXPR_SKIP */
+ as = &state->steps[skip_step_off];
+ as->d.jsonexpr_skip.jump_coercion = coercion_step_off;
+ as->d.jsonexpr_skip.jump_passing_args = passing_args_step_off;
- jsestate->default_on_empty = !jexpr->on_empty ? NULL :
- ExecInitExpr((Expr *) jexpr->on_empty->default_expr,
- state->parent);
+ /* EEOP_JSONEXPR_BEHAVIOR */
+ as = &state->steps[behavior_step_off];
+ as->d.jsonexpr_behavior.jump_onerror_default = onerror_default_step_off;
+ as->d.jsonexpr_behavior.jump_onempty_default = onempty_default_step_off;
+ as->d.jsonexpr_behavior.jump_coercion = coercion_step_off;
+ as->d.jsonexpr_behavior.jump_skip_coercion = coercion_step_off + 1;
- jsestate->default_on_error =
- ExecInitExpr((Expr *) jexpr->on_error->default_expr,
- state->parent);
+ /* EEOP_JSONEXPR_COERCION */
+ as = &state->steps[coercion_step_off];
+ as->d.jsonexpr_coercion.jump_coercion_error = behavior_step_off;
+ as->d.jsonexpr_coercion.jump_coercion_done = coercion_step_off + 1;
+ /*
+ * EEOP_JUMP steps added after ON EMPTY and ON ERROR default
+ * expression should jump to the current step address.
+ */
+ foreach(lc, adjust_jumps)
+ {
+ as = &state->steps[lfirst_int(lc)];
+
+ as->d.jump.jumpdone = state->steps_len;
+ }
+
+ /*
+ * Set if we must use a sub-transaction around path evaluation
+ * that can be aborted on error instead of aborting the parent
+ * query.
+ */
+ jsestate->needSubtrans =
+ ExecEvalJsonNeedsSubTransaction(jexpr,
+ true /* coerce */);
+
+ /*
+ * Set if coercion, which runs separately from path evaluation
+ * and whose errors must be caught and handled per ON ERROR
+ * behavior, must use a sub-transaction.
+ */
+ jsestate->coercionNeedSubtrans = !throwErrors;
+
+ /*
+ * Set RETURNING type's input function used by
+ * ExecEvalJsonExprCoercion().
+ */
if (jexpr->omit_quotes ||
(jexpr->result_coercion && jexpr->result_coercion->via_io))
{
Oid typinput;
+ FmgrInfo *finfo;
/* lookup the result type's input function */
getTypeInputInfo(jexpr->returning->typid, &typinput,
&jsestate->input.typioparam);
- fmgr_info(typinput, &jsestate->input.func);
+ finfo = palloc0(sizeof(FmgrInfo));
+ fmgr_info(typinput, finfo);
+ jsestate->input.finfo = finfo;
}
- jsestate->args = NIL;
+ /*
+ * Initialize coercion expression(s). These can't be inlined
+ * into the parent ExprState, because of the need to use a
+ * sub-transaction around their evaluation.
+ */
+ if (jexpr->result_coercion && jexpr->result_coercion->expr)
+ jsestate->result_coercion =
+ ExecInitExprWithCaseValue((Expr *)
+ jexpr->result_coercion,
+ state->parent,
+ resv, resnull);
+ if (jexpr->coercions)
+ jsestate->coercions =
+ ExecInitExprWithCaseValue((Expr *) jexpr->coercions,
+ state->parent,
+ resv, resnull);
- forboth(argexprlc, jexpr->passing_values,
- argnamelc, jexpr->passing_names)
- {
- Expr *argexpr = (Expr *) lfirst(argexprlc);
- String *argname = lfirst_node(String, argnamelc);
- JsonPathVariableEvalContext *var = palloc(sizeof(*var));
+ break;
+ }
- var->name = pstrdup(argname->sval);
- var->typid = exprType((Node *) argexpr);
- var->typmod = exprTypmod((Node *) argexpr);
- var->estate = ExecInitExpr(argexpr, state->parent);
- var->econtext = NULL;
- var->mcxt = NULL;
- var->evaluated = false;
- var->value = (Datum) 0;
- var->isnull = true;
+ case T_JsonCoercion:
+ {
+ JsonCoercion *coercion = castNode(JsonCoercion, node);
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
- jsestate->args =
- lappend(jsestate->args, var);
- }
+ /*
+ * Only ever get here for JsonExpr.result_expression and the
+ * caller should have checked this.
+ */
+ Assert(coercion->expr);
- jsestate->cache = NULL;
+ /*
+ * Push a step to read the value provided by the parent
+ * JsonExpr via a CaseTestExpr.
+ */
+ scratch.opcode = EEOP_CASE_TESTVAL;
+ scratch.d.casetest.value = state->innermost_caseval;
+ scratch.d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, &scratch);
- if (jexpr->coercions)
- {
- JsonCoercion **coercion;
- struct JsonCoercionState *cstate;
- Datum *caseval;
- bool *casenull;
+ /* Push step(s) to compute coercion->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
+
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) coercion->expr, state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
+
+ break;
+ }
+
+ case T_JsonItemCoercions:
+ {
+ JsonItemCoercions *coercions = castNode(JsonItemCoercions,
+ node);
+ JsonItemCoercionsState *jcstate =
+ palloc0(sizeof(JsonItemCoercionsState));
+ JsonCoercion **coercion;
+ JsonItemCoercionState *cstate;
+ ExprEvalStep *as;
+ int json_item_coercion_step_id;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
- jsestate->coercion_expr =
- palloc(sizeof(*jsestate->coercion_expr));
+ /*
+ * Push a step to read the value provided by the parent
+ * JsonExpr via a CaseTestExpr.
+ */
+ scratch.opcode = EEOP_CASE_TESTVAL;
+ scratch.d.casetest.value = state->innermost_caseval;
+ scratch.d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, &scratch);
- caseval = &jsestate->coercion_expr->value;
- casenull = &jsestate->coercion_expr->isnull;
+ /* Push the control step. */
+ scratch.opcode = EEOP_JSON_ITEM_COERCION;
+ scratch.d.jsonexpr_item_coercion.jcstate = jcstate;
+ json_item_coercion_step_id = state->steps_len;
+ ExprEvalPushStep(state, &scratch);
+ /* Will set jump_skip_coercion target address below. */
- for (cstate = &jsestate->coercions.null,
- coercion = &jexpr->coercions->null;
- coercion <= &jexpr->coercions->composite;
- coercion++, cstate++)
+ /*
+ * Now push the steps of individual coercion's expression, if
+ * needed.
+ */
+ for (cstate = &jcstate->null,
+ coercion = &coercions->null;
+ coercion <= &coercions->composite;
+ coercion++, cstate++)
+ {
+ cstate->coercion = *coercion;
+ if (cstate->coercion && cstate->coercion->expr)
{
- cstate->coercion = *coercion;
- cstate->estate = *coercion ?
- ExecInitExprWithCaseValue((Expr *) (*coercion)->expr,
- state->parent,
- caseval, casenull) : NULL;
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
+
+ cstate->jump_eval_expr = state->steps_len;
+
+ /* Push step(s) to compute (*coercion)->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
+
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) cstate->coercion->expr,
+ state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
}
+ else
+ cstate->jump_eval_expr = -1;
+
+ /*
+ * Emit JUMP step to jump to end of coercions code,
+ * because evaluating the coercion expression gives the
+ * final result and there's nothing more to do.
+ */
+ scratch.opcode = EEOP_JUMP;
+
+ /*
+ * Remember JUMP step address to set the actual jump
+ * target address below.
+ */
+ adjust_jumps = lappend_int(adjust_jumps,
+ state->steps_len);
+ ExprEvalPushStep(state, &scratch);
+ }
+
+ /*
+ * Adjust the jump target address in the control step and all
+ * the jumps we added in the above loop to jump here.
+ */
+ as = &state->steps[json_item_coercion_step_id];
+ as->d.jsonexpr_item_coercion.jump_skip_item_coercion = state->steps_len;
+ foreach(lc, adjust_jumps)
+ {
+ int jump_step_id = lfirst_int(lc);
+
+ as = &state->steps[jump_step_id];
+ as->d.jump.jumpdone = state->steps_len;
}
- ExprEvalPushStep(state, &scratch);
break;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 723770fda0..964b0088a4 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -158,6 +158,15 @@ static TupleDesc get_cached_rowtype(Oid type_id, int32 typmod,
bool *changed);
static void ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, bool checkisnull);
+static Datum ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null);
+typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
+ Datum item, bool *resnull, void *p, bool *error);
+static Datum ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
+ ExprContext *econtext,
+ Datum res, bool *resnull,
+ void *p, bool *error, bool subtrans);
+static Datum ExecEvalJsonCoercion(ExprEvalStep *op, ExprContext *econtext,
+ Datum res, bool *isNull, void *p, bool *error);
/* fast-path evaluation functions */
static Datum ExecJustInnerVar(ExprState *state, ExprContext *econtext, bool *isnull);
@@ -490,7 +499,11 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SUBPLAN,
&&CASE_EEOP_JSON_CONSTRUCTOR,
&&CASE_EEOP_IS_JSON,
- &&CASE_EEOP_JSONEXPR,
+ &&CASE_EEOP_JSONEXPR_SKIP,
+ &&CASE_EEOP_JSONEXPR_PATH,
+ &&CASE_EEOP_JSONEXPR_BEHAVIOR,
+ &&CASE_EEOP_JSONEXPR_COERCION,
+ &&CASE_EEOP_JSON_ITEM_COERCION,
&&CASE_EEOP_AGG_STRICT_DESERIALIZE,
&&CASE_EEOP_AGG_DESERIALIZE,
&&CASE_EEOP_AGG_STRICT_INPUT_CHECK_ARGS,
@@ -1817,13 +1830,37 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
- EEO_CASE(EEOP_JSONEXPR)
+ EEO_CASE(EEOP_JSONEXPR_PATH)
{
/* too complex for an inline implementation */
ExecEvalJson(state, op, econtext);
EEO_NEXT();
}
+ EEO_CASE(EEOP_JSONEXPR_SKIP)
+ {
+ ExecEvalJsonExprSkip(state, op);
+ EEO_JUMP(op->d.jsonexpr_skip.jumpdone);
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_BEHAVIOR)
+ {
+ ExecEvalJsonExprBehavior(state, op);
+ EEO_JUMP(op->d.jsonexpr_behavior.jumpdone);
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_COERCION)
+ {
+ ExecEvalJsonExprCoercion(state, op, econtext);
+ EEO_JUMP(op->d.jsonexpr_coercion.jumpdone);
+ }
+
+ EEO_CASE(EEOP_JSON_ITEM_COERCION)
+ {
+ ExecEvalJsonExprItemCoercion(state, op);
+ EEO_JUMP(op->d.jsonexpr_item_coercion.jumpdone);
+ }
+
EEO_CASE(EEOP_LAST)
{
/* unreachable */
@@ -4602,8 +4639,7 @@ ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
* Evaluate a JSON error/empty behavior result.
*/
static Datum
-ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
- ExprState *default_estate, bool *is_null)
+ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null)
{
*is_null = false;
@@ -4628,7 +4664,9 @@ ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
return (Datum) 0;
case JSON_BEHAVIOR_DEFAULT:
- return ExecEvalExpr(default_estate, econtext, is_null);
+ /* Always handled in the caller. */
+ Assert(false);
+ return (Datum) 0;
default:
elog(ERROR, "unrecognized SQL/JSON behavior %d", behavior->btype);
@@ -4640,17 +4678,18 @@ ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
* Evaluate a coercion of a JSON item to the target type.
*/
static Datum
-ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
- Datum res, bool *isNull, void *p, bool *error)
+ExecEvalJsonCoercion(ExprEvalStep *op, ExprContext *econtext,
+ Datum res, bool *isNull, void *p, bool *error)
{
- ExprState *estate = p;
- JsonExprState *jsestate;
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ ExprState *estate = post_eval->coercion;
+
+ post_eval->coercion_error = false;
if (estate) /* coerce using specified expression */
return ExecEvalExpr(estate, econtext, isNull);
- jsestate = op->d.jsonexpr.jsestate;
-
if (jsestate->jsexpr->op != JSON_EXISTS_OP)
{
JsonCoercion *coercion = jsestate->jsexpr->result_coercion;
@@ -4664,7 +4703,7 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
/* strip quotes and call typinput function */
char *str = *isNull ? NULL : JsonbUnquote(jb);
- return InputFunctionCall(&jsestate->input.func, str,
+ return InputFunctionCall(jsestate->input.finfo, str,
jsestate->input.typioparam,
jexpr->returning->typmod);
}
@@ -4672,17 +4711,17 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
return json_populate_type(res, JSONBOID,
jexpr->returning->typid,
jexpr->returning->typmod,
- &jsestate->cache,
+ &post_eval->cache,
econtext->ecxt_per_query_memory,
isNull);
}
- if (jsestate->result_expr)
+ /* Coerce with jexpr->result_coercion if there's one */
+ if (jsestate->result_coercion)
{
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *isNull;
-
- res = ExecEvalExpr(jsestate->result_expr, econtext, isNull);
+ *op->resvalue = res;
+ *op->resnull = *isNull;
+ return ExecEvalExpr(jsestate->result_coercion, econtext, isNull);
}
return res;
@@ -4717,17 +4756,30 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
if (!var)
return -1;
- if (!var->evaluated)
+ /*
+ * When belonging to a JsonExpr, path variables are computed with the
+ * JsonExpr's ExprState (var->estate is NULL), so don't need to be computed
+ * here. In some other cases, such as when the path variables belonging
+ * to a JsonTable instead, those variables must be evaluated on their own,
+ * without the enclosing JsonExpr itself needing to be evaluated, so must
+ * be handled here.
+ */
+ if (var->estate && !var->evaluated)
{
MemoryContext oldcxt = var->mcxt ?
MemoryContextSwitchTo(var->mcxt) : NULL;
+ Assert(var->econtext != NULL);
var->value = ExecEvalExpr(var->estate, var->econtext, &var->isnull);
var->evaluated = true;
if (oldcxt)
MemoryContextSwitchTo(oldcxt);
}
+ else
+ {
+ Assert(var->evaluated);
+ }
if (var->isnull)
{
@@ -4741,19 +4793,84 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
return id;
}
+/*
+ * Return a coercion among those given in 'coercions' for given
+ * JSON item.
+ */
+JsonCoercion *
+ExecGetJsonItemCoercion(JsonbValue *item, JsonItemCoercions *coercions)
+{
+ if (item->type == jbvBinary &&
+ JsonContainerIsScalar(item->val.binary.data))
+ {
+ JsonbValue buf;
+ bool res PG_USED_FOR_ASSERTS_ONLY;
+
+ res = JsonbExtractScalar(item->val.binary.data, &buf);
+ item = &buf;
+ Assert(res);
+ }
+
+ /* get coercion state reference and datum of the corresponding SQL type */
+ switch (item->type)
+ {
+ case jbvNull:
+ return coercions->null;
+
+ case jbvString:
+ return coercions->string;
+
+ case jbvNumeric:
+ return coercions->numeric;
+
+ case jbvBool:
+ return coercions->boolean;
+
+ case jbvDatetime:
+ switch (item->val.datetime.typid)
+ {
+ case DATEOID:
+ return coercions->date;
+ case TIMEOID:
+ return coercions->time;
+ case TIMETZOID:
+ return coercions->timetz;
+ case TIMESTAMPOID:
+ return coercions->timestamp;
+ case TIMESTAMPTZOID:
+ return coercions->timestamptz;
+ default:
+ elog(ERROR, "unexpected jsonb datetime type oid %u",
+ item->val.datetime.typid);
+ }
+ break;
+
+ case jbvArray:
+ case jbvObject:
+ case jbvBinary:
+ return coercions->composite;
+
+ default:
+ elog(ERROR, "unexpected jsonb value type %d", item->type);
+ }
+
+ Assert(false);
+ return NULL;
+}
+
/*
* Prepare SQL/JSON item coercion to the output type. Returned a datum of the
* corresponding SQL type and a pointer to the coercion state.
*/
Datum
-ExecPrepareJsonItemCoercion(JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pcoercion)
+ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pcoercion)
{
- struct JsonCoercionState *coercion;
+ JsonbValue *item = DatumGetJsonbValueP(itemval);
+ JsonItemCoercionState *coercion;
Datum res;
- JsonbValue buf;
+ JsonbValue buf;
if (item->type == jbvBinary &&
JsonContainerIsScalar(item->val.binary.data))
@@ -4832,9 +4949,6 @@ ExecPrepareJsonItemCoercion(JsonbValue *item,
return res;
}
-typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
- Datum item, bool *resnull, void *p, bool *error);
-
static Datum
ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
ExprContext *econtext,
@@ -4905,7 +5019,6 @@ typedef struct
{
JsonPath *path;
bool *error;
- bool coercionInSubtrans;
} ExecEvalJsonExprContext;
static Datum
@@ -4916,16 +5029,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
ExecEvalJsonExprContext *cxt = pcxt;
JsonPath *path = cxt->path;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
- ExprState *estate = NULL;
- bool empty = false;
+ bool *empty = &post_eval->empty;
Datum res = (Datum) 0;
switch (jexpr->op)
{
case JSON_QUERY_OP:
- res = JsonPathQuery(item, path, jexpr->wrapper, &empty, error,
- jsestate->args);
+ res = JsonPathQuery(item, path, jexpr->wrapper, empty, error,
+ pre_eval->args);
if (error && *error)
{
*resnull = true;
@@ -4936,17 +5050,20 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
case JSON_VALUE_OP:
{
- struct JsonCoercionState *jcstate;
- JsonbValue *jbv = JsonPathValue(item, path, &empty, error,
- jsestate->args);
+ JsonCoercion *coercion;
+ JsonbValue *jbv = JsonPathValue(item, path, empty, error,
+ pre_eval->args);
if (error && *error)
+ {
+ *resnull = true;
return (Datum) 0;
+ }
if (!jbv) /* NULL or empty */
break;
- Assert(!empty);
+ Assert(!*empty);
*resnull = false;
@@ -4959,19 +5076,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
break;
}
- /* Use coercion from SQL/JSON item type to the output type */
- res = ExecPrepareJsonItemCoercion(jbv,
- jsestate->jsexpr->returning,
- &jsestate->coercions,
- &jcstate);
-
- if (jcstate->coercion &&
- (jcstate->coercion->via_io ||
- jcstate->coercion->via_populate))
+ /*
+ * Error out if no cast exists to coerce SQL/JSON item to the
+ * the output type
+ */
+ coercion = ExecGetJsonItemCoercion(jbv, jsestate->jsexpr->coercions);
+ if (coercion && (coercion->via_io || coercion->via_populate))
{
if (error)
{
*error = true;
+ *resnull = true;
return (Datum) 0;
}
@@ -4983,32 +5098,42 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_SQL_JSON_ITEM_CANNOT_BE_CAST_TO_TARGET_TYPE),
errmsg("SQL/JSON item cannot be cast to target type")));
}
- else if (!jcstate->estate)
- return res; /* no coercion */
-
- /* coerce using specific expression */
- estate = jcstate->estate;
- jsestate->coercion_expr->value = res;
- jsestate->coercion_expr->isnull = *resnull;
+ else
+ {
+ /*
+ * Coerce using a specific coercion (JsonItemCoercion).
+ * Note we are passing the JsobbValue as is, without
+ * converting it to Jsonb, because
+ * ExecPrepareJsonItemCoercion() expects the JSON item
+ * to be in that format.
+ */
+ res = JsonbValuePGetDatum(jbv);
+ post_eval->coercion = jsestate->coercions;
+ }
break;
}
case JSON_EXISTS_OP:
{
bool exists = JsonPathExists(item, path,
- jsestate->args,
+ pre_eval->args,
error);
*resnull = error && *error;
res = BoolGetDatum(exists);
- if (!jsestate->result_expr)
+ if (jexpr->result_coercion == NULL)
+ {
+ /* No coercion needed */
+ post_eval->coercion_done = true;
return res;
-
- /* coerce using result expression */
- estate = jsestate->result_expr;
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *resnull;
+ }
+ else
+ {
+ /* coerce using result expression */
+ Assert(jsestate->result_coercion);
+ post_eval->coercion = jsestate->result_coercion;
+ }
break;
}
@@ -5021,7 +5146,11 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
return (Datum) 0;
}
- if (empty)
+ /*
+ * If the ON EMPTY behavior is to cause an error, do so here. Other
+ * behaviors will be handled by the caller.
+ */
+ if (*empty)
{
Assert(jexpr->on_empty); /* it is not JSON_EXISTS */
@@ -5037,29 +5166,13 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_NO_SQL_JSON_ITEM),
errmsg("no SQL/JSON item")));
}
-
- if (jexpr->on_empty->btype == JSON_BEHAVIOR_DEFAULT)
-
- /*
- * Execute DEFAULT expression as a coercion expression, because
- * its result is already coerced to the target type.
- */
- estate = jsestate->default_on_empty;
- else
- /* Execute ON EMPTY behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_empty,
- jsestate->default_on_empty,
- resnull);
}
- return ExecEvalJsonExprSubtrans(ExecEvalJsonExprCoercion, op, econtext,
- res, resnull, estate, error,
- cxt->coercionInSubtrans);
+ return res;
}
bool
-ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *coercions)
+ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce)
{
if (jsexpr->on_error->btype == JSON_BEHAVIOR_ERROR)
return false;
@@ -5067,12 +5180,164 @@ ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
if (jsexpr->op == JSON_EXISTS_OP && !jsexpr->result_coercion)
return false;
- if (!coercions)
+ if (!coerce)
return true;
return false;
}
+/* Skip calling ExecEvalJson() on a JsonExpr? */
+void
+ExecEvalJsonExprSkip(ExprState *state, ExprEvalStep *op)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_skip.jsestate;
+ int *jump_to = &op->d.jsonexpr_skip.jumpdone;
+
+ /*
+ * Skip if either of the input expressions has turned out to be NULL,
+ * though do execute domain checks for NULLs, which are handled by the
+ * coercion step.
+ */
+ if (jsestate->pre_eval.formatted_expr.isnull ||
+ jsestate->pre_eval.pathspec.isnull)
+ {
+ *op->resvalue = 0;
+ *op->resnull = true;
+ Assert(jsestate->post_eval.coercion == NULL);
+ /* A signal to the coercion step to not use a sub-trancaction. */
+ jsestate->post_eval.coercion_error = true;
+ *jump_to = op->d.jsonexpr_skip.jump_coercion;
+ }
+ else
+ {
+ /*
+ * Go evaluate the PASSING args if any and subsequently JSON path
+ * itself.
+ */
+ *jump_to = op->d.jsonexpr_skip.jump_passing_args;
+ }
+}
+
+void
+ExecEvalJsonExprBehavior(ExprState *state, ExprEvalStep *op)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_behavior.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ JsonBehavior *behavior = NULL;
+ bool error = (post_eval->error || post_eval->coercion_error);
+ int *jump_to = &op->d.jsonexpr_behavior.jumpdone;
+
+ /*
+ * Set if the coercion step, if will run afterwards, should use a sub-
+ * transaction to be aborted on error instead of throwing the error.
+ * Don't use a sub-transaction if coercing the ON ERROR expression,
+ * though.
+ *
+ * XXX - doesn't that violate ON ERROR behavior? Actually, that is
+ * how it behaved even before this expression eval logic rewrite!
+ */
+ post_eval->coercion_use_subtrans =
+ (jsestate->coercionNeedSubtrans && !error);
+
+ /*
+ * If no error or the JSON item is not empty, directly go to the coercion
+ * step to coerce the item as is, or skip the coercion step if the item is
+ * already coerced by ExecEvalJson().
+ */
+ if (!error && !post_eval->empty)
+ {
+ *jump_to = !post_eval->coercion_done ?
+ op->d.jsonexpr_behavior.jump_coercion :
+ op->d.jsonexpr_behavior.jump_skip_coercion;
+ return;
+ }
+
+ if (error)
+ {
+ behavior = jsestate->jsexpr->on_error;
+ *jump_to = op->d.jsonexpr_behavior.jump_onerror_default;
+ }
+ else if (post_eval->empty)
+ {
+ behavior = jsestate->jsexpr->on_empty;
+ *jump_to = op->d.jsonexpr_behavior.jump_onempty_default;
+ }
+
+ Assert(behavior);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate value and go
+ * to the coercion step.
+ */
+ if (behavior->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ *op->resvalue = ExecEvalJsonBehavior(behavior, op->resnull);
+
+ if (post_eval->coercion_error)
+ {
+ post_eval->coercion_done = false;
+ post_eval->coercion = NULL;
+ }
+
+ *jump_to = op->d.jsonexpr_behavior.jump_coercion;
+ }
+
+ /*
+ * Else evaluate the default ON ERROR or ON EMPTY expression, with no
+ * coercion needed afterwards given that the expression is already
+ * coerced appropriately in the parser.
+ */
+
+ Assert(op->d.jsonexpr_behavior.jumpdone >= 0);
+}
+
+/*
+ * Apply coercion to a JSON item using either JsonExpr.result_coercion or one
+ * of JsonExpr.coercions, possibly using a sub-transaction.
+ */
+void
+ExecEvalJsonExprCoercion(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ int *jump_to = &op->d.jsonexpr_coercion.jumpdone;
+
+ *op->resvalue =
+ ExecEvalJsonExprSubtrans(ExecEvalJsonCoercion, op, econtext,
+ *op->resvalue, op->resnull, NULL,
+ &post_eval->coercion_error,
+ post_eval->coercion_use_subtrans);
+ if (post_eval->coercion_error)
+ *jump_to = op->d.jsonexpr_coercion.jump_coercion_error;
+ else
+ *jump_to = op->d.jsonexpr_coercion.jump_coercion_done;
+}
+
+/*
+ * Apply a coercion to a JSON item, choosing one among those present in
+ * JsonItemCoercions.
+ */
+void
+ExecEvalJsonExprItemCoercion(ExprState *state, ExprEvalStep *op)
+{
+ JsonItemCoercionsState *jcstate = op->d.jsonexpr_item_coercion.jcstate;
+ JsonItemCoercionState *cstate = NULL;
+ int *jump_to = &op->d.jsonexpr_item_coercion.jumpdone;
+
+ *op->resvalue =
+ ExecPrepareJsonItemCoercion(*op->resvalue, jcstate, &cstate);
+ if (cstate->coercion && cstate->coercion->expr)
+ {
+ Assert(cstate->jump_eval_expr >= 0);
+ *jump_to = cstate->jump_eval_expr;
+ }
+ else
+ {
+ /* Skip over all of the steps added for this JsonItemCoercions. */
+ *jump_to = op->d.jsonexpr_item_coercion.jump_skip_item_coercion;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEvalJson
* ----------------------------------------------------------------
@@ -5082,64 +5347,30 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
{
ExecEvalJsonExprContext cxt;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
Datum item;
Datum res = (Datum) 0;
JsonPath *path;
- ListCell *lc;
- bool error = false;
- bool needSubtrans;
bool throwErrors = jexpr->on_error->btype == JSON_BEHAVIOR_ERROR;
*op->resnull = true; /* until we get a result */
*op->resvalue = (Datum) 0;
- if (jsestate->formatted_expr->isnull || jsestate->pathspec->isnull)
- {
- /* execute domain checks for NULLs */
- (void) ExecEvalJsonExprCoercion(op, econtext, res, op->resnull,
- NULL, NULL);
+ item = pre_eval->formatted_expr.value;
+ path = DatumGetJsonPathP(pre_eval->pathspec.value);
- Assert(*op->resnull);
- return;
- }
-
- item = jsestate->formatted_expr->value;
- path = DatumGetJsonPathP(jsestate->pathspec->value);
-
- /* reset JSON path variable contexts */
- foreach(lc, jsestate->args)
- {
- JsonPathVariableEvalContext *var = lfirst(lc);
-
- var->econtext = econtext;
- var->evaluated = false;
- }
-
- needSubtrans = ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ /* Reset JsonExprPostEvalState for this evaluation. */
+ memset(post_eval, 0, sizeof(*post_eval));
cxt.path = path;
- cxt.error = throwErrors ? NULL : &error;
- cxt.coercionInSubtrans = !needSubtrans && !throwErrors;
- Assert(!needSubtrans || cxt.error);
+ cxt.error = throwErrors ? NULL : &post_eval->error;
+ Assert(!jsestate->needSubtrans || cxt.error);
res = ExecEvalJsonExprSubtrans(ExecEvalJsonExpr, op, econtext, item,
op->resnull, &cxt, cxt.error,
- needSubtrans);
-
- if (error)
- {
- /* Execute ON ERROR behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_error,
- jsestate->default_on_error,
- op->resnull);
-
- /* result is already coerced in DEFAULT behavior case */
- if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
- res = ExecEvalJsonExprCoercion(op, econtext, res,
- op->resnull,
- NULL, NULL);
- }
+ jsestate->needSubtrans);
*op->resvalue = res;
}
diff --git a/src/backend/jit/llvm/llvmjit_expr.c b/src/backend/jit/llvm/llvmjit_expr.c
index b6b6512ef1..c76302fba3 100644
--- a/src/backend/jit/llvm/llvmjit_expr.c
+++ b/src/backend/jit/llvm/llvmjit_expr.c
@@ -2359,12 +2359,229 @@ llvm_compile_expr(ExprState *state)
LLVMBuildBr(b, opblocks[opno + 1]);
break;
- case EEOP_JSONEXPR:
+ case EEOP_JSONEXPR_PATH:
build_EvalXFunc(b, mod, "ExecEvalJson",
v_state, op, v_econtext);
LLVMBuildBr(b, opblocks[opno + 1]);
break;
+ case EEOP_JSONEXPR_SKIP:
+ {
+ LLVMValueRef v_jumpaddrp;
+ LLVMValueRef v_jumpaddr;
+
+ /*
+ * Call ExecEvalJsonExprSkip() to decide if JSON path
+ * evaluation can be skipped and and set
+ * jsonexpr_skip.jumpdone if so.
+ */
+ build_EvalXFunc(b, mod, "ExecEvalJsonExprSkip",
+ v_state, op);
+ /* Read jsonexpr_behavior.jumpdone into v_jumpaddr. */
+ v_jumpaddrp = l_ptr_const(&op->d.jsonexpr_skip.jumpdone, l_ptr(TypeSizeT));
+ v_jumpaddr = LLVMBuildLoad(b, v_jumpaddrp, "");
+
+ /*
+ * Jump to coercion step if
+ * jsonexpr_skip.jumpdone == jsonexpr_skip.jump_coercion,
+ * which signifies skipping of JSON path evaluation, else
+ * to the next step which must point to the steps to
+ * evaluate PASSING args, if any, or to the JSON path
+ * evaluation.
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_skip.jump_coercion),
+ ""),
+ opblocks[op->d.jsonexpr_skip.jump_coercion],
+ opblocks[opno + 1]);
+ break;
+ }
+
+ case EEOP_JSONEXPR_BEHAVIOR:
+ {
+ LLVMValueRef v_jumpaddrp;
+ LLVMValueRef v_jumpaddr;
+ LLVMBasicBlockRef b_jump_skip_coercion,
+ b_jump_onerror_default;
+
+ /*
+ * Call ExecEvalJsonExprBehavior() to decide if ON EMPTY or
+ * ON ERROR behavior must be invoked depending on what JSON
+ * path evaluation returned. This sets
+ * jsonexpr_behavior.jumpdone to an address as described
+ * below.
+ */
+ build_EvalXFunc(b, mod, "ExecEvalJsonExprBehavior",
+ v_state, op);
+ /* Read jsonexpr_behavior.jumpdone into v_jumpaddr. */
+ v_jumpaddrp = l_ptr_const(&op->d.jsonexpr_behavior.jumpdone, l_ptr(TypeSizeT));
+ v_jumpaddr = LLVMBuildLoad(b, v_jumpaddrp, "");
+
+ b_jump_skip_coercion =
+ l_bb_before_v(opblocks[opno + 1],
+ "op.%d.jsonexpr_behavior_jump_skip_coercion", opno);
+ b_jump_onerror_default =
+ l_bb_before_v(opblocks[opno + 1],
+ "op.%d.jsonexpr_behavior_jump_onerror_default", opno);
+
+ /*
+ * Jump to coercion step if
+ * jsonexpr_behavior.jumpdone == jsonexpr_behavior.jump_coercion,
+ * else to the block that checks whether to skip coercion.
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_behavior.jump_coercion),
+ ""),
+ opblocks[op->d.jsonexpr_behavior.jump_coercion],
+ b_jump_skip_coercion);
+
+ /*
+ * Block that checks whether to skip coercion.
+ *
+ * Jump to skip coercion if
+ * jsonexpr_behavior.jumpdone == jsonexpr_behavior.jump_skip_coercion,
+ * else to check whether the ON ERROR or the ON EMPTY
+ * default expression must be evaluated instead.
+ */
+ LLVMPositionBuilderAtEnd(b, b_jump_skip_coercion);
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_behavior.jump_skip_coercion),
+ ""),
+ opblocks[op->d.jsonexpr_behavior.jump_skip_coercion],
+ b_jump_onerror_default);
+
+ /*
+ * Block that checks whether to evaluate the ON ERROR
+ * default expression.
+ *
+ * Jump to evaluate ON ERROR default expression if
+ * jsonexpr_behavior.jumpdone == jsonexpr_behavior.jump_onerror_default,
+ * else jump to evaluate ON EMPTY default expression.
+ */
+ LLVMPositionBuilderAtEnd(b, b_jump_onerror_default);
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_behavior.jump_onerror_default),
+ ""),
+ opblocks[op->d.jsonexpr_behavior.jump_onerror_default],
+ opblocks[op->d.jsonexpr_behavior.jump_onempty_default]);
+ break;
+ }
+ case EEOP_JSONEXPR_COERCION:
+ {
+ LLVMValueRef v_jumpaddrp;
+ LLVMValueRef v_jumpaddr;
+
+ /*
+ * Call ExecEvalJsonExprCoercion() to evaluate appropriate
+ * coercion. That will set jsonexpr_coercion.jumpdone to
+ * an address as described below.
+ */
+ build_EvalXFunc(b, mod, "ExecEvalJsonExprCoercion",
+ v_state, op, v_econtext);
+ /* Read jsonexpr_coercion.jumpdone into v_jumpaddr. */
+ v_jumpaddrp = l_ptr_const(&op->d.jsonexpr_coercion.jumpdone, l_ptr(TypeSizeT));
+ v_jumpaddr = LLVMBuildLoad(b, v_jumpaddrp, "");
+
+ /*
+ * Jump to handle coercion error if
+ * jsonexpr_coercion.jumpdone == jsonexpr_coercion.jump_coercion_error,
+ * else to the step after coercion (coercion done!).
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(op->d.jsonexpr_coercion.jump_coercion_error),
+ ""),
+ opblocks[op->d.jsonexpr_coercion.jump_coercion_error],
+ opblocks[op->d.jsonexpr_coercion.jump_coercion_done]);
+ break;
+ }
+ case EEOP_JSON_ITEM_COERCION:
+ {
+ JsonItemCoercionsState *jcstate = op->d.jsonexpr_item_coercion.jcstate;
+ JsonItemCoercionState *cstate;
+ LLVMValueRef v_jumpaddrp;
+ LLVMValueRef v_jumpaddr;
+ int n_coercions = (int) (&jcstate->composite - &jcstate->null) + 1;
+ int i;
+ LLVMBasicBlockRef *b_coercions;
+
+ /*
+ * Call ExecEvalJsonExprItemCoercion() to inspect the JSON
+ * item obtained by path evaluation and pick the coercion
+ * to apply accordingly. That will set
+ * jsonexpr_item_coercion.jumpdone to an address as
+ * described below.
+ */
+ build_EvalXFunc(b, mod, "ExecEvalJsonExprItemCoercion",
+ v_state, op);
+ /* Read jsonexpr_item_coercion.jumpdone into v_jumpaddr. */
+ v_jumpaddrp = l_ptr_const(&op->d.jsonexpr_item_coercion.jumpdone, l_ptr(TypeSizeT));
+ v_jumpaddr = LLVMBuildLoad(b, v_jumpaddrp, "");
+
+ /*
+ * Will create a block for each coercion below to check
+ * whether to evaluate the coercion's expression if there's
+ * one or to skip to the end if not.
+ */
+ b_coercions = palloc((n_coercions + 1) * sizeof(LLVMBasicBlockRef));
+ for (i = 0; i < n_coercions + 1; i++)
+ b_coercions[i] =
+ l_bb_before_v(opblocks[opno + 1],
+ "op.%d.json_item_coercion.%d",
+ opno, i);
+
+ /* Jump to check first coercion */
+ LLVMBuildBr(b, b_coercions[0]);
+
+ /* Add conditional branches for individual coercion's expressions */
+ for (cstate = &jcstate->null, i = 0;
+ cstate <= &jcstate->composite;
+ cstate++, i++)
+ {
+ /* Block for this coercion */
+ LLVMPositionBuilderAtEnd(b, b_coercions[i]);
+
+ /*
+ * Jump to evaluate the coercion's expression if
+ * jsonexpr_item_coercion.jumpdone == cstate->jump_eval_expr,
+ * that is, if cstate->jump_eval_expr is valid (skip to end
+ * if not), else to the next coercion's block.
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_jumpaddr,
+ l_int32_const(cstate->jump_eval_expr),
+ ""),
+ cstate->jump_eval_expr >= 0 ?
+ opblocks[cstate->jump_eval_expr] :
+ opblocks[op->d.jsonexpr_item_coercion.jump_skip_item_coercion],
+ b_coercions[i + 1]);
+ }
+
+ /*
+ * A placeholder block that the last coercion's block might
+ * jump to, which unconditionally jumps to end of
+ * coercions.
+ */
+ LLVMPositionBuilderAtEnd(b, b_coercions[i]);
+ LLVMBuildBr(b, opblocks[op->d.jsonexpr_item_coercion.jump_skip_item_coercion]);
+ break;
+ }
case EEOP_LAST:
Assert(false);
break;
diff --git a/src/backend/jit/llvm/llvmjit_types.c b/src/backend/jit/llvm/llvmjit_types.c
index b2bda86889..34e13b4a7e 100644
--- a/src/backend/jit/llvm/llvmjit_types.c
+++ b/src/backend/jit/llvm/llvmjit_types.c
@@ -133,6 +133,10 @@ void *referenced_functions[] =
ExecEvalXmlExpr,
ExecEvalJsonConstructor,
ExecEvalJsonIsPredicate,
+ ExecEvalJsonExprSkip,
+ ExecEvalJsonExprBehavior,
+ ExecEvalJsonExprCoercion,
+ ExecEvalJsonExprItemCoercion,
ExecEvalJson,
MakeExpandedObjectReadOnlyInternal,
slot_getmissingattrs,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..9c02218155 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -901,7 +901,11 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
{
JsonExpr *jsexpr = (JsonExpr *) node;
- if (ExecEvalJsonNeedsSubTransaction(jsexpr, NULL))
+ /*
+ * XXX - don't really know why it makes sense to ignore the coercion
+ * part here.
+ */
+ if (ExecEvalJsonNeedsSubTransaction(jsexpr, false /* coerce */))
{
context->max_hazard = PROPARALLEL_UNSAFE;
return true;
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index 1e3f1bbee8..b2e6b813b8 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -244,7 +244,11 @@ typedef enum ExprEvalOp
EEOP_SUBPLAN,
EEOP_JSON_CONSTRUCTOR,
EEOP_IS_JSON,
- EEOP_JSONEXPR,
+ EEOP_JSONEXPR_SKIP,
+ EEOP_JSONEXPR_PATH,
+ EEOP_JSONEXPR_BEHAVIOR,
+ EEOP_JSONEXPR_COERCION,
+ EEOP_JSON_ITEM_COERCION,
/* aggregation related nodes */
EEOP_AGG_STRICT_DESERIALIZE,
@@ -682,12 +686,71 @@ typedef struct ExprEvalStep
JsonIsPredicate *pred; /* original expression node */
} is_json;
- /* for EEOP_JSONEXPR */
+ /* for EEOP_JSONEXPR_PATH */
struct
{
struct JsonExprState *jsestate;
} jsonexpr;
+ /* for EEOP_JSONEXPR_SKIP */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_coercion;
+ int jump_passing_args;
+
+ /*
+ * ExecEvalJsonExprSkip() sets this to one of the above
+ * addresses.
+ */
+ int jumpdone;
+ } jsonexpr_skip;
+
+ /* for EEOP_JSONEXPR_BEHAVIOR */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_onerror_default;
+ int jump_onempty_default;
+ int jump_coercion;
+ int jump_skip_coercion;
+
+ /*
+ * ExecEvalJsonExprBehavior() sets this to one of the above
+ * addresses.
+ */
+ int jumpdone;
+ } jsonexpr_behavior;
+
+ /* for EEOP_JSONEXPR_COERCION */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+ int jump_coercion_error;
+ int jump_coercion_done;
+
+ /*
+ * ExecEvalJsonExprCoercion() sets this to one of the above
+ * addresses.
+ */
+ int jumpdone;
+ } jsonexpr_coercion;
+
+ struct
+ {
+ struct JsonItemCoercionsState *jcstate;
+ int jump_skip_item_coercion;
+
+ /*
+ * ExecEvalJsonExprItemCoercion() sets this to either the "skip"
+ * address above or to the coercion expression's address given in
+ * one of the JsonItemCoercionStates found in 'jcstate'.
+ */
+ int jumpdone;
+ } jsonexpr_item_coercion;
} d;
} ExprEvalStep;
@@ -747,49 +810,117 @@ typedef struct JsonConstructorExprState
int nargs;
} JsonConstructorExprState;
+/*
+ * Information computed before evaluating a JsonExpr expression.
+ */
+typedef struct JsonExprPreEvalState
+{
+ /* value/isnull for JsonExpr.formatted_expr */
+ NullableDatum formatted_expr;
+
+ /* value/isnull for JsonExpr.pathspec */
+ NullableDatum pathspec;
+
+ /* JsonPathVariableEvalContext entries for JsonExpr.passing_values */
+ List *args;
+} JsonExprPreEvalState;
+
+/*
+ * State for some post-JsonExpr-evaluation processing steps that gets filled
+ * in JsonExpr evaluation.
+ */
+typedef struct JsonExprPostEvalState
+{
+ /* Is JSON item empty? */
+ bool empty;
+
+ /* Did JSON item evaluation cause an error? */
+ bool error;
+
+ /* Did coercion evaluation cause an error? */
+ bool coercion_error;
+
+ /* Has the result been coerced properly? */
+ bool coercion_done;
+
+ /* Use a sub-transaction when evaluating the coercion */
+ bool coercion_use_subtrans;
+
+ /* Cache for json_populate_type() called for coercion in some cases */
+ void *cache;
+
+ /*
+ * State for evaluating a JSON item coercion. Points to either
+ * JsonExprState.coercions or JsonExprState.result_coercion; might be
+ * set to NULL if it is to be left for ExecEvalJsonCoercion() to decide
+ * how to perform the coercion.
+ */
+ ExprState *coercion;
+} JsonExprPostEvalState;
+
/* EEOP_JSONEXPR state, too big to inline */
typedef struct JsonExprState
{
JsonExpr *jsexpr; /* original expression node */
+ JsonExprPreEvalState pre_eval;
+ JsonExprPostEvalState post_eval;
+
+ /*
+ * Should use a sub-transaction for path evaluation and subsequent
+ * coercion evaluation, if any?
+ */
+ bool needSubtrans;
+ bool coercionNeedSubtrans;
+
struct
{
- FmgrInfo func; /* typinput function for output type */
+ FmgrInfo *finfo; /* typinput function for output type */
Oid typioparam;
} input; /* I/O info for output type */
- NullableDatum
- *formatted_expr, /* formatted context item value */
- *res_expr, /* result item */
- *coercion_expr, /* input for JSON item coercion */
- *pathspec; /* path specification value */
+ /*
+ * Either of the following two is used by ExecEvalJsonCoercion() to apply
+ * coercion to the final result if needed.
+ */
+ ExprState *result_coercion;
+ ExprState *coercions;
+} JsonExprState;
- ExprState *result_expr; /* coerced to output type */
- ExprState *default_on_empty; /* ON EMPTY DEFAULT expression */
- ExprState *default_on_error; /* ON ERROR DEFAULT expression */
- List *args; /* passing arguments */
+/*
+ * State for a given member of JsonItemCoercions.
+ */
+typedef struct JsonItemCoercionState
+{
+ /* Expression used to evaluate the coercion */
+ JsonCoercion *coercion;
- void *cache; /* cache for json_populate_type() */
+ /* ExprEvalStep to compute this coercion's expression */
+ int jump_eval_expr;
+} JsonItemCoercionState;
- struct JsonCoercionsState
- {
- struct JsonCoercionState
- {
- JsonCoercion *coercion; /* coercion expression */
- ExprState *estate; /* coercion expression state */
- } null,
- string,
- numeric ,
- boolean,
- date,
- time,
- timetz,
- timestamp,
- timestamptz,
- composite;
- } coercions; /* states for coercion from SQL/JSON item
- * types directly to the output type */
-} JsonExprState;
+/*
+ * State for evaluating the coercion for a given JSON item using one of
+ * the following coercions.
+ *
+ * Note that while ExecInitExprRec() for JsonItemCoercions will initialize
+ * ExprEvalSteps for all of the members that need it, only one will get run
+ * during a given evaluation of the enclosing JsonExpr depending on the type
+ * of the result JSON item.
+ */
+typedef struct JsonItemCoercionsState
+{
+ JsonItemCoercionState null;
+ JsonItemCoercionState string;
+ JsonItemCoercionState numeric;
+ JsonItemCoercionState boolean;
+ JsonItemCoercionState date;
+ JsonItemCoercionState time;
+ JsonItemCoercionState timetz;
+ JsonItemCoercionState timestamp;
+ JsonItemCoercionState timestamptz;
+ JsonItemCoercionState composite;
+} JsonItemCoercionsState;
/* functions in execExpr.c */
extern void ExprEvalPushStep(ExprState *es, const ExprEvalStep *s);
@@ -850,14 +981,18 @@ extern void ExecEvalSysVar(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, TupleTableSlot *slot);
extern void ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
+extern void ExecEvalJsonExprSkip(ExprState *state, ExprEvalStep *op);
+extern void ExecEvalJsonExprBehavior(ExprState *state, ExprEvalStep *op);
+extern void ExecEvalJsonExprCoercion(ExprState *state, ExprEvalStep *op,
+ ExprContext *econtext);
+extern void ExecEvalJsonExprItemCoercion(ExprState *state, ExprEvalStep *op);
extern void ExecEvalJson(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
-extern Datum ExecPrepareJsonItemCoercion(struct JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pjcstate);
-extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *);
+JsonCoercion *ExecGetJsonItemCoercion(struct JsonbValue *item, JsonItemCoercions *coercions);
+extern Datum ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pjcstate);
+extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool coerce);
extern Datum ExecEvalExprPassingCaseValue(ExprState *estate,
ExprContext *econtext, bool *isnull,
Datum caseval_datum,
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index bae466b523..6bdd9f5121 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -69,8 +69,10 @@ typedef enum
/* Convenience macros */
#define DatumGetJsonbP(d) ((Jsonb *) PG_DETOAST_DATUM(d))
+#define DatumGetJsonbValueP(d) ((JsonbValue *) DatumGetPointer(d))
#define DatumGetJsonbPCopy(d) ((Jsonb *) PG_DETOAST_DATUM_COPY(d))
#define JsonbPGetDatum(p) PointerGetDatum(p)
+#define JsonbValuePGetDatum(p) PointerGetDatum(p)
#define PG_GETARG_JSONB_P(x) DatumGetJsonbP(PG_GETARG_DATUM(x))
#define PG_GETARG_JSONB_P_COPY(x) DatumGetJsonbPCopy(PG_GETARG_DATUM(x))
#define PG_RETURN_JSONB_P(x) PG_RETURN_POINTER(x)
--
2.35.3
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-07-29 18:27 ` Andrew Dunstan <andrew@dunslane.net>
2022-08-01 23:27 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 75+ messages in thread
From: Andrew Dunstan @ 2022-07-29 18:27 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Andres Freund <andres@anarazel.de>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-07-27 We 04:01, Amit Langote wrote:
> On Fri, Jul 22, 2022 at 2:49 PM Amit Langote <amitlangote09@gmail.com> wrote:
>> On Fri, Jul 22, 2022 at 1:13 PM David Rowley <dgrowleyml@gmail.com> wrote:
>>> BTW, I was working on code inside llvm_compile_expr() a few days ago
>>> and I thought I'd gotten the new evaluation steps I was adding correct
>>> as it worked fine with jit_above_cost=0, but on further testing, it
>>> crashed with jit_inline_above_cost=0. Might be worth doing both to see
>>> if everything works as intended.
>> Thanks for the pointer.
>>
>> So I didn't see things going bust on re-testing with all
>> jit_*_above_cost parameters set to 0, so maybe the
>> llvm_compile_expression() additions are alright.
> Here's an updated version of the patch, with mostly cosmetic changes.
> In particular, I added comments describing the new llvm_compile_expr()
> blobs.
>
Andres,
this work has been done in response to a complaint from you. Does this
address your concerns satisfactorily?
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-29 18:27 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-08-01 23:27 ` Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 75+ messages in thread
From: Andres Freund @ 2022-08-01 23:27 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-07-29 14:27:36 -0400, Andrew Dunstan wrote:
> this work has been done in response to a complaint from you. Does this
> address your concerns satisfactorily?
Will look. Was on vacation for the last two weeks...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-08-02 00:39 ` Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
1 sibling, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-08-02 00:39 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-07-27 17:01:13 +0900, Amit Langote wrote:
> Here's an updated version of the patch, with mostly cosmetic changes.
> In particular, I added comments describing the new llvm_compile_expr()
> blobs.
- I've asked a couple times before: Why do we need space for every possible
datatype at once in JsonItemCoercions? Can there be multiple "concurrent"
coercions in process?
The whole coercion stuff just seems incredibly clunky (in a slightly
different shape before this patch). ExecEvalJsonExprItemCoercion() calls
ExecPrepareJsonItemCoercion(), which gets a pointer to one of the per-type
elements in JsonItemCoercionsState, dispatching on the type of the json
object. Then we later call ExecGetJsonItemCoercion() (via a convoluted
path), which again will dispatch on the type (extracting the json object
again afaics!), to then somehow eventually get the coerced value.
I cannot make any sense of this. This code should not have been committed
in this state.
- Looks like there's still some recursive expression states, namely
JsonExprState->{result_coercion, coercions}?
- Looks like the JsonExpr code in ExecInitExprRec() is big enough to
potentially benefit from splitting out into a separate function?
- looks like JsonExprPostEvalState could be moved to execExprInterp.c?
- I ran the patch against LLVM 14 built with assertions enabled, and it
triggers an assertion failure:
#3 0x00007f75d165c242 in __GI___assert_fail (
assertion=0x7f75c278d511 "getOperand(0)->getType() == getOperand(1)->getType() && \"Both operands to ICmp instruction are not of the same type!\"",
file=0x7f75c2780366 "/home/andres/src/llvm-project-14/llvm/include/llvm/IR/Instructions.h", line=1192,
function=0x7f75c27d9dcc "void llvm::ICmpInst::AssertOK()") at assert.c:101
#4 0x00007f75c2b9b25c in llvm::ICmpInst::AssertOK (this=0x55e019290ca0) at /home/andres/src/llvm-project-14/llvm/include/llvm/IR/Instructions.h:1191
#5 0x00007f75c2b9b0ea in llvm::ICmpInst::ICmpInst (this=0x55e019290ca0, pred=llvm::CmpInst::ICMP_EQ, LHS=0x55e019290c10, RHS=0x55e01928ce80, NameStr="")
at /home/andres/src/llvm-project-14/llvm/include/llvm/IR/Instructions.h:1246
#6 0x00007f75c2b93c99 in llvm::IRBuilderBase::CreateICmp (this=0x55e0192894f0, P=llvm::CmpInst::ICMP_EQ, LHS=0x55e019290c10, RHS=0x55e01928ce80, Name="")
at /home/andres/src/llvm-project-14/llvm/include/llvm/IR/IRBuilder.h:2202
#7 0x00007f75c2c1bc5d in LLVMBuildICmp (B=0x55e0192894f0, Op=LLVMIntEQ, LHS=0x55e019290c10, RHS=0x55e01928ce80, Name=0x7f75d0d24cbc "")
at /home/andres/src/llvm-project-14/llvm/lib/IR/Core.cpp:3927
#8 0x00007f75d0d20b1f in llvm_compile_expr (state=0x55e019201380) at /home/andres/src/postgresql/src/backend/jit/llvm/llvmjit_expr.c:2392
...
#19 0x000055e0184c16d4 in exec_simple_query (query_string=0x55e01912f6e0 "SELECT JSON_EXISTS(NULL::jsonb, '$');") at /home/andres/src/postgresql/src/backend/tcop/postgres.c:1204
this triggers easily interactively - which is nice because that allows to
dump the types:
p getOperand(0)->getType()->dump() -> prints i64
p getOperand(1)->getType()->dump() -> prints i32
The immediate issue is that you're setting v_jumpaddrp up as a pointer to a
pointer to size_t - but then compare it to i32.
I first was confused why the code tries to load the jump target
dynamically. But then I saw that the interpreted code sets it dynamically -
why? That's completely unnecessary overhead afaics? There's just two
possible jump targets, no?
- why is EvalJsonPathVar() in execExprInterp.c, when it's only ever called
from within jsonpath_exec.c?
- s/JsobbValue/JsonbValue/
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-08-02 03:05 ` Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 75+ messages in thread
From: Amit Langote @ 2022-08-02 03:05 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
Thanks for looking into this.
On Tue, Aug 2, 2022 at 9:39 AM Andres Freund <andres@anarazel.de> wrote:
> On 2022-07-27 17:01:13 +0900, Amit Langote wrote:
> > Here's an updated version of the patch, with mostly cosmetic changes.
> > In particular, I added comments describing the new llvm_compile_expr()
> > blobs.
>
> - I've asked a couple times before: Why do we need space for every possible
> datatype at once in JsonItemCoercions? Can there be multiple "concurrent"
> coercions in process?
This topic has been a head-scratcher for me too from the beginning,
but I've since come to understand (convince myself) that we do need
the coercions for all possible types, because we don't know the type
of the JSON item that's going to pop out of the main JSON path
expression until we've run it through the JSON path executor that
ExecEvalJson() invokes. So, it's not possible to statically assign
the coercion. I am not really sure if different coercions may be used
in the same query over multiple evaluations of the same JSON path
expression, but maybe that's also possible.
> The whole coercion stuff just seems incredibly clunky (in a slightly
> different shape before this patch). ExecEvalJsonExprItemCoercion() calls
> ExecPrepareJsonItemCoercion(), which gets a pointer to one of the per-type
> elements in JsonItemCoercionsState, dispatching on the type of the json
> object. Then we later call ExecGetJsonItemCoercion() (via a convoluted
> path), which again will dispatch on the type (extracting the json object
> again afaics!), to then somehow eventually get the coerced value.
I think it might be possible to make this a bit simpler, by not
leaving anything coercion-related in ExecEvalJsonExpr(). I left some
pieces there, because I thought the error of not finding an
appropriate coercion must be thrown right away as the code in
ExecEvalJsonExpr() does after calling ExecGetJsonItemCoercion().
ExecPrepareJsonItemCoercion() is called later when it's time to
actually evaluate the coercion. If we move the error path to
ExecPrepareJsonItemCoercion(), both ExecGetJsonItemCoercion() and the
error path code in ExecEvalJsonExpr() will be unnecessary. I will
give that a try.
> - Looks like there's still some recursive expression states, namely
> JsonExprState->{result_coercion, coercions}?
So, the problem with inlining coercion evaluation into the main parent
JsonExpr's is that it needs to be wrapped in a sub-transaction to
catch any errors and return NULL instead. I don't know a way to wrap
ExprEvalStep evaluation in a sub-transaction to achieve that effect.
> - Looks like the JsonExpr code in ExecInitExprRec() is big enough to
> potentially benefit from splitting out into a separate function?
Thought about it too, so will do.
> - looks like JsonExprPostEvalState could be moved to execExprInterp.c?
OK, will give that a try.
> - I ran the patch against LLVM 14 built with assertions enabled, and it
> triggers an assertion failure:
>
> #3 0x00007f75d165c242 in __GI___assert_fail (
> assertion=0x7f75c278d511 "getOperand(0)->getType() == getOperand(1)->getType() && \"Both operands to ICmp instruction are not of the same type!\"",
> file=0x7f75c2780366 "/home/andres/src/llvm-project-14/llvm/include/llvm/IR/Instructions.h", line=1192,
> function=0x7f75c27d9dcc "void llvm::ICmpInst::AssertOK()") at assert.c:101
> #4 0x00007f75c2b9b25c in llvm::ICmpInst::AssertOK (this=0x55e019290ca0) at /home/andres/src/llvm-project-14/llvm/include/llvm/IR/Instructions.h:1191
> #5 0x00007f75c2b9b0ea in llvm::ICmpInst::ICmpInst (this=0x55e019290ca0, pred=llvm::CmpInst::ICMP_EQ, LHS=0x55e019290c10, RHS=0x55e01928ce80, NameStr="")
> at /home/andres/src/llvm-project-14/llvm/include/llvm/IR/Instructions.h:1246
> #6 0x00007f75c2b93c99 in llvm::IRBuilderBase::CreateICmp (this=0x55e0192894f0, P=llvm::CmpInst::ICMP_EQ, LHS=0x55e019290c10, RHS=0x55e01928ce80, Name="")
> at /home/andres/src/llvm-project-14/llvm/include/llvm/IR/IRBuilder.h:2202
> #7 0x00007f75c2c1bc5d in LLVMBuildICmp (B=0x55e0192894f0, Op=LLVMIntEQ, LHS=0x55e019290c10, RHS=0x55e01928ce80, Name=0x7f75d0d24cbc "")
> at /home/andres/src/llvm-project-14/llvm/lib/IR/Core.cpp:3927
> #8 0x00007f75d0d20b1f in llvm_compile_expr (state=0x55e019201380) at /home/andres/src/postgresql/src/backend/jit/llvm/llvmjit_expr.c:2392
> ...
> #19 0x000055e0184c16d4 in exec_simple_query (query_string=0x55e01912f6e0 "SELECT JSON_EXISTS(NULL::jsonb, '$');") at /home/andres/src/postgresql/src/backend/tcop/postgres.c:1204
>
> this triggers easily interactively - which is nice because that allows to
> dump the types:
>
> p getOperand(0)->getType()->dump() -> prints i64
> p getOperand(1)->getType()->dump() -> prints i32
>
> The immediate issue is that you're setting v_jumpaddrp up as a pointer to a
> pointer to size_t - but then compare it to i32.
Ooh, thanks for letting me know. So maybe I am missing some
llvmjist_emit.h/type.c infrastructure to read an int32 value
(jumpdone) out of an int32 pointer (&jumpdone)?
> I first was confused why the code tries to load the jump target
> dynamically. But then I saw that the interpreted code sets it dynamically -
> why? That's completely unnecessary overhead afaics? There's just two
> possible jump targets, no?
Hmm, I looked at the code for other expressions that jump, especially
CASE WHEN, but they use ready-made EEOP_JUMP_IF_* steps, which can be
added statically. I thought we can't use them in this case, because
the conditions are very ad-hoc, like if the JSON path computation
returned an "empty" item or if the "error" flag was set during that
computation, etc.
> - why is EvalJsonPathVar() in execExprInterp.c, when it's only ever called
> from within jsonpath_exec.c?
Hadn't noticed that because the patch didn't really have to touch it,
but yes, maybe it makes sense to move it there.
> - s/JsobbValue/JsonbValue/
Oops, will fix.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-08-02 15:00 ` Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-08-02 15:00 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-08-02 12:05:55 +0900, Amit Langote wrote:
> On Tue, Aug 2, 2022 at 9:39 AM Andres Freund <andres@anarazel.de> wrote:
> > On 2022-07-27 17:01:13 +0900, Amit Langote wrote:
> > > Here's an updated version of the patch, with mostly cosmetic changes.
> > > In particular, I added comments describing the new llvm_compile_expr()
> > > blobs.
> >
> > - I've asked a couple times before: Why do we need space for every possible
> > datatype at once in JsonItemCoercions? Can there be multiple "concurrent"
> > coercions in process?
>
> This topic has been a head-scratcher for me too from the beginning,
> but I've since come to understand (convince myself) that we do need
> the coercions for all possible types, because we don't know the type
> of the JSON item that's going to pop out of the main JSON path
> expression until we've run it through the JSON path executor that
> ExecEvalJson() invokes. So, it's not possible to statically assign
> the coercion.
Sure. But that doesn't mean we have to have memory for every possible type *at
the same time*.
> I am not really sure if different coercions may be used
> in the same query over multiple evaluations of the same JSON path
> expression, but maybe that's also possible.
Even if the type can change, I don't think that means we need to have space
for multiple types at the same time - there can't be multiple coercions
happening at the same time, otherwise there could be two coercions of the same
type as well. So we don't need memory for every coercion type.
>
> > The whole coercion stuff just seems incredibly clunky (in a slightly
> > different shape before this patch). ExecEvalJsonExprItemCoercion() calls
> > ExecPrepareJsonItemCoercion(), which gets a pointer to one of the per-type
> > elements in JsonItemCoercionsState, dispatching on the type of the json
> > object. Then we later call ExecGetJsonItemCoercion() (via a convoluted
> > path), which again will dispatch on the type (extracting the json object
> > again afaics!), to then somehow eventually get the coerced value.
>
> I think it might be possible to make this a bit simpler, by not
> leaving anything coercion-related in ExecEvalJsonExpr().
Honestly, this code seems like it should just be rewritten from scratch.
> I left some pieces there, because I thought the error of not finding an
> appropriate coercion must be thrown right away as the code in
> ExecEvalJsonExpr() does after calling ExecGetJsonItemCoercion().
>
> ExecPrepareJsonItemCoercion() is called later when it's time to
> actually evaluate the coercion. If we move the error path to
> ExecPrepareJsonItemCoercion(), both ExecGetJsonItemCoercion() and the
> error path code in ExecEvalJsonExpr() will be unnecessary. I will
> give that a try.
Why do we need the separation of prepare and then evaluation? They're executed
straight after each other?
> > - Looks like there's still some recursive expression states, namely
> > JsonExprState->{result_coercion, coercions}?
>
> So, the problem with inlining coercion evaluation into the main parent
> JsonExpr's is that it needs to be wrapped in a sub-transaction to
> catch any errors and return NULL instead. I don't know a way to wrap
> ExprEvalStep evaluation in a sub-transaction to achieve that effect.
But we don't need to wrap arbitrary evaluation in a subtransaction - afaics
the coercion calls a single function, not an arbitrary expression?
> Ooh, thanks for letting me know. So maybe I am missing some
> llvmjist_emit.h/type.c infrastructure to read an int32 value
> (jumpdone) out of an int32 pointer (&jumpdone)?
No, you just need to replace l_ptr(TypeSizeT) with l_ptr(LLVMInt32Type()).
> > I first was confused why the code tries to load the jump target
> > dynamically. But then I saw that the interpreted code sets it dynamically -
> > why? That's completely unnecessary overhead afaics? There's just two
> > possible jump targets, no?
>
> Hmm, I looked at the code for other expressions that jump, especially
> CASE WHEN, but they use ready-made EEOP_JUMP_IF_* steps, which can be
> added statically. I thought we can't use them in this case, because
> the conditions are very ad-hoc, like if the JSON path computation
> returned an "empty" item or if the "error" flag was set during that
> computation, etc.
The minimal fix would be to return the jump target from the function, and then
jump to that. That at least avoids the roundtrip to memory you have right now.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-08-04 08:01 ` Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 75+ messages in thread
From: Amit Langote @ 2022-08-04 08:01 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On Wed, Aug 3, 2022 at 12:00 AM Andres Freund <andres@anarazel.de> wrote:
> On 2022-08-02 12:05:55 +0900, Amit Langote wrote:
> > On Tue, Aug 2, 2022 at 9:39 AM Andres Freund <andres@anarazel.de> wrote:
> > > On 2022-07-27 17:01:13 +0900, Amit Langote wrote:
> > > > Here's an updated version of the patch, with mostly cosmetic changes.
> > > > In particular, I added comments describing the new llvm_compile_expr()
> > > > blobs.
> > >
> > > - I've asked a couple times before: Why do we need space for every possible
> > > datatype at once in JsonItemCoercions? Can there be multiple "concurrent"
> > > coercions in process?
> >
> > This topic has been a head-scratcher for me too from the beginning,
> > but I've since come to understand (convince myself) that we do need
> > the coercions for all possible types, because we don't know the type
> > of the JSON item that's going to pop out of the main JSON path
> > expression until we've run it through the JSON path executor that
> > ExecEvalJson() invokes. So, it's not possible to statically assign
> > the coercion.
>
> Sure. But that doesn't mean we have to have memory for every possible type *at
> the same time*.
>
> > I am not really sure if different coercions may be used
> > in the same query over multiple evaluations of the same JSON path
> > expression, but maybe that's also possible.
>
> Even if the type can change, I don't think that means we need to have space
> for multiple types at the same time - there can't be multiple coercions
> happening at the same time, otherwise there could be two coercions of the same
> type as well. So we don't need memory for every coercion type.
Do you find it unnecessary to statically allocate memory for
JsonItemCoercionState for each possible coercion, as in the following
struct definition:
typedef struct JsonItemCoercionsState
{
JsonItemCoercionState null;
JsonItemCoercionState string;
JsonItemCoercionState numeric;
JsonItemCoercionState boolean;
JsonItemCoercionState date;
JsonItemCoercionState time;
JsonItemCoercionState timetz;
JsonItemCoercionState timestamp;
JsonItemCoercionState timestamptz;
JsonItemCoercionState composite;
} JsonItemCoercionsState;
A given JsonItemCoercionState (note singular Coercion) contains:
typedef struct JsonItemCoercionState
{
/* Expression used to evaluate the coercion */
JsonCoercion *coercion;
/* ExprEvalStep to compute this coercion's expression */
int jump_eval_expr;
} JsonItemCoercionState;
jump_eval_expr above is the address in JsonItemCoercions'
ExprState.steps of the 1st ExprEvalStep corresponding to
coercion->expr. IIUC, all ExprEvalSteps needed to evaluate an
expression and its children must be allocated statically in
ExecInitExprRec(), and none on-the-fly as needed. So, this considers
all coercions and allocates states of all statically.
> > > The whole coercion stuff just seems incredibly clunky (in a slightly
> > > different shape before this patch). ExecEvalJsonExprItemCoercion() calls
> > > ExecPrepareJsonItemCoercion(), which gets a pointer to one of the per-type
> > > elements in JsonItemCoercionsState, dispatching on the type of the json
> > > object. Then we later call ExecGetJsonItemCoercion() (via a convoluted
> > > path), which again will dispatch on the type (extracting the json object
> > > again afaics!), to then somehow eventually get the coerced value.
> >
> > I think it might be possible to make this a bit simpler, by not
> > leaving anything coercion-related in ExecEvalJsonExpr().
>
> Honestly, this code seems like it should just be rewritten from scratch.
Based on what I wrote above, please let me know if I've misunderstood
your concerns about over-allocation of coercion state. I can try to
rewrite one more time if I know what this should look like instead.
> > I left some pieces there, because I thought the error of not finding an
> > appropriate coercion must be thrown right away as the code in
> > ExecEvalJsonExpr() does after calling ExecGetJsonItemCoercion().
> >
> > ExecPrepareJsonItemCoercion() is called later when it's time to
> > actually evaluate the coercion. If we move the error path to
> > ExecPrepareJsonItemCoercion(), both ExecGetJsonItemCoercion() and the
> > error path code in ExecEvalJsonExpr() will be unnecessary. I will
> > give that a try.
>
> Why do we need the separation of prepare and then evaluation? They're executed
> straight after each other?
ExecPrepareJsonItemCoercion() is a helper routine to choose the
coercion and extract the Datum out of the JsonbValue produced by the
EEOP_JSONEXPR_PATH step to feed to the coercion expression's
ExprEvalStep. The coercion evaluation will be done by jumping to said
step in ExecInterpExpr().
> > > - Looks like there's still some recursive expression states, namely
> > > JsonExprState->{result_coercion, coercions}?
> >
> > So, the problem with inlining coercion evaluation into the main parent
> > JsonExpr's is that it needs to be wrapped in a sub-transaction to
> > catch any errors and return NULL instead. I don't know a way to wrap
> > ExprEvalStep evaluation in a sub-transaction to achieve that effect.
>
> But we don't need to wrap arbitrary evaluation in a subtransaction - afaics
> the coercion calls a single function, not an arbitrary expression?
It can do EEOP_IOCOERCE for example, and the input/output function may
cause an error depending on what comes out of the JSON blob.
IIUC, those errors need to be caught to satisfy some SQL/JSON spec.
> > Ooh, thanks for letting me know. So maybe I am missing some
> > llvmjist_emit.h/type.c infrastructure to read an int32 value
> > (jumpdone) out of an int32 pointer (&jumpdone)?
>
> No, you just need to replace l_ptr(TypeSizeT) with l_ptr(LLVMInt32Type()).
OK, thanks.
> > > I first was confused why the code tries to load the jump target
> > > dynamically. But then I saw that the interpreted code sets it dynamically -
> > > why? That's completely unnecessary overhead afaics? There's just two
> > > possible jump targets, no?
> >
> > Hmm, I looked at the code for other expressions that jump, especially
> > CASE WHEN, but they use ready-made EEOP_JUMP_IF_* steps, which can be
> > added statically. I thought we can't use them in this case, because
> > the conditions are very ad-hoc, like if the JSON path computation
> > returned an "empty" item or if the "error" flag was set during that
> > computation, etc.
>
> The minimal fix would be to return the jump target from the function, and then
> jump to that. That at least avoids the roundtrip to memory you have right now.
You mean like this:
LLVMValueRef v_args[2];
LLVMValueRef v_ret;
/*
* Call ExecEvalJsonExprSkip() to decide if JSON path
* evaluation can be skipped. This returns the step
* address to jump to.
*/
v_args[0] = v_state;
v_args[1] = l_ptr_const(op, l_ptr(StructExprEvalStep));
v_ret = LLVMBuildCall(b,
llvm_pg_func(mod,
"ExecEvalJsonExprSkip"),
params, lengthof(params), "");
Actually, this is how I had started, but never figured out how to jump
to the address in v_ret. As in, how to extract the plain C int32
value that is the jump address from v_ret, an LLVMValueRef, to use in
the following statement:
LLVMBuildBr(b, opblocks[<int32-in-v_ret>]);
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-08-05 20:36 ` Andres Freund <andres@anarazel.de>
2022-08-08 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
0 siblings, 2 replies; 75+ messages in thread
From: Andres Freund @ 2022-08-05 20:36 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
I tried to look into some of the questions from Amit, but I have e.g. no idea
what exactly the use of subtransactions tries to achieve - afaics 1a36bc9dba8
is the first patch to introduce needing to evaluate parts expressions in a
subtransaction - but there's not a single comment explaining why.
It's really hard to understand the new json code. It's a substantial amount of
new infrastructure, without any design documentation that I can find. And it's
not like it's just code that's equivalent to nearby stuff. To me this falls
way below our standards and I think it's *months* of work to fix.
Even just the expresion evaluation code: EvalJsonPathVar(),
ExecEvalJsonConstructor(), ExecEvalJsonExpr(), ExecEvalJson(). There's one
layer of subtransactions in one of the paths in ExecEvalJsonExpr(), another in
ExecEvalJson(). Some paths of ExecEvalJsonExpr() go through subtransactions,
others don't.
It's one thing for a small set of changes to be of this quality. But this is
pretty large - a quick summing of diffstat ends up with about 17k lines added,
of which ~2.5k are docs, ~4.8k are tests.
On 2022-08-04 17:01:48 +0900, Amit Langote wrote:
> On Wed, Aug 3, 2022 at 12:00 AM Andres Freund <andres@anarazel.de> wrote:
> > > > The whole coercion stuff just seems incredibly clunky (in a slightly
> > > > different shape before this patch). ExecEvalJsonExprItemCoercion() calls
> > > > ExecPrepareJsonItemCoercion(), which gets a pointer to one of the per-type
> > > > elements in JsonItemCoercionsState, dispatching on the type of the json
> > > > object. Then we later call ExecGetJsonItemCoercion() (via a convoluted
> > > > path), which again will dispatch on the type (extracting the json object
> > > > again afaics!), to then somehow eventually get the coerced value.
> > >
> > > I think it might be possible to make this a bit simpler, by not
> > > leaving anything coercion-related in ExecEvalJsonExpr().
> >
> > Honestly, this code seems like it should just be rewritten from scratch.
>
> Based on what I wrote above, please let me know if I've misunderstood
> your concerns about over-allocation of coercion state. I can try to
> rewrite one more time if I know what this should look like instead.
I don't know. I don't understand the design of what needs to have error
handling, and what not.
> > > > I first was confused why the code tries to load the jump target
> > > > dynamically. But then I saw that the interpreted code sets it dynamically -
> > > > why? That's completely unnecessary overhead afaics? There's just two
> > > > possible jump targets, no?
> > >
> > > Hmm, I looked at the code for other expressions that jump, especially
> > > CASE WHEN, but they use ready-made EEOP_JUMP_IF_* steps, which can be
> > > added statically. I thought we can't use them in this case, because
> > > the conditions are very ad-hoc, like if the JSON path computation
> > > returned an "empty" item or if the "error" flag was set during that
> > > computation, etc.
> >
> > The minimal fix would be to return the jump target from the function, and then
> > jump to that. That at least avoids the roundtrip to memory you have right now.
>
> You mean like this:
>
> LLVMValueRef v_args[2];
> LLVMValueRef v_ret;
>
> /*
> * Call ExecEvalJsonExprSkip() to decide if JSON path
> * evaluation can be skipped. This returns the step
> * address to jump to.
> */
> v_args[0] = v_state;
> v_args[1] = l_ptr_const(op, l_ptr(StructExprEvalStep));
> v_ret = LLVMBuildCall(b,
> llvm_pg_func(mod,
> "ExecEvalJsonExprSkip"),
> params, lengthof(params), "");
>
> Actually, this is how I had started, but never figured out how to jump
> to the address in v_ret. As in, how to extract the plain C int32
> value that is the jump address from v_ret, an LLVMValueRef, to use in
> the following statement:
>
> LLVMBuildBr(b, opblocks[<int32-in-v_ret>]);
We could make that work, but even keeping it more similar to your current
code, you're already dealing with a variable jump target. Only that you load
it from a variable, instead of the function return type. So you could just
v_ret instead of v_jumpaddr, and your code would be simpler and faster.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-08-08 07:38 ` Amit Langote <amitlangote09@gmail.com>
1 sibling, 0 replies; 75+ messages in thread
From: Amit Langote @ 2022-08-08 07:38 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Jonathan S. Katz <jkatz@postgresql.org>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi Andres,
On Sat, Aug 6, 2022 at 5:37 AM Andres Freund <andres@anarazel.de> wrote:
> On 2022-08-04 17:01:48 +0900, Amit Langote wrote:
> > On Wed, Aug 3, 2022 at 12:00 AM Andres Freund <andres@anarazel.de> wrote:
> > > Honestly, this code seems like it should just be rewritten from scratch.
> >
> > Based on what I wrote above, please let me know if I've misunderstood
> > your concerns about over-allocation of coercion state. I can try to
> > rewrite one more time if I know what this should look like instead.
>
> I don't know. I don't understand the design of what needs to have error
> handling, and what not.
AFAIK, there are two things that the JSON path code considers can
cause an error when evaluating a JsonExpr:
* Actual JSON path evaluation in ExecEvalJsonExpr(), because it
invokes JsonPath*() family of functions defined in jsonpath_exec.c,
which can possibly cause an error. Actually, I looked again today as
to what goes on in them and it seems there is a throwErrors variable
being used to catch and prevent any errors found by the JSON path
machinery itself, and it has the same value as the throwErrors
variable in ExecEvalJsonExpr(). Given that the latter is set per the
ON ERROR specification (throw errors or return NULL / a default
expression in lieu), maybe this part doesn't really need a
sub-transaction. To check, I took off the sub-transaction around this
part and can see that no tests fail.
* Evaluating coercion expression in ExecEvalJsonExprCoercion(), which
involves passing a user-specified expression through either
EEOP_IOCOERCE or json_populate_type(), both of which can cause errors
that are not suppressible as those in jsonpath_exec.c are. So, this
part does need a sub-transaction to satisfy the ON ERROR behavior. To
check, I took out the sub-transaction around the coercion evaluation,
and some tests in jsob_sqljson do indeed fail, like this, for example:
SELECT JSON_VALUE(jsonb '"aaa"', '$' RETURNING int);
- json_value
-------------
-
-(1 row)
-
+ERROR: invalid input syntax for type integer: "aaa"
Note that both the JSON path expression and the coercion would run as
part of the one EEOP_JSONEXPR ExecEvalStep before this patch and thus
would be wrapped under the same sub-transaction, even if only the
latter apparently needs it.
With this patch, I tried to keep that same behavior, but because the
coercion evaluation has now been broken out into its own step, it must
use another sub-transaction, given that the same sub-transaction can
no longer wrap both things. But given my finding that the JSON path
expression doesn't really need one, maybe the new EEOP_JSONEXPR_PATH
step can run without one, while keeping it for the new
EEOP_JSONEXPR_COERCION step.
> > > > > I first was confused why the code tries to load the jump target
> > > > > dynamically. But then I saw that the interpreted code sets it dynamically -
> > > > > why? That's completely unnecessary overhead afaics? There's just two
> > > > > possible jump targets, no?
> > > >
> > > > Hmm, I looked at the code for other expressions that jump, especially
> > > > CASE WHEN, but they use ready-made EEOP_JUMP_IF_* steps, which can be
> > > > added statically. I thought we can't use them in this case, because
> > > > the conditions are very ad-hoc, like if the JSON path computation
> > > > returned an "empty" item or if the "error" flag was set during that
> > > > computation, etc.
> > >
> > > The minimal fix would be to return the jump target from the function, and then
> > > jump to that. That at least avoids the roundtrip to memory you have right now.
> >
> > You mean like this:
> >
> > LLVMValueRef v_args[2];
> > LLVMValueRef v_ret;
> >
> > /*
> > * Call ExecEvalJsonExprSkip() to decide if JSON path
> > * evaluation can be skipped. This returns the step
> > * address to jump to.
> > */
> > v_args[0] = v_state;
> > v_args[1] = l_ptr_const(op, l_ptr(StructExprEvalStep));
> > v_ret = LLVMBuildCall(b,
> > llvm_pg_func(mod,
> > "ExecEvalJsonExprSkip"),
> > params, lengthof(params), "");
> >
> > Actually, this is how I had started, but never figured out how to jump
> > to the address in v_ret. As in, how to extract the plain C int32
> > value that is the jump address from v_ret, an LLVMValueRef, to use in
> > the following statement:
> >
> > LLVMBuildBr(b, opblocks[<int32-in-v_ret>]);
>
> We could make that work, but even keeping it more similar to your current
> code, you're already dealing with a variable jump target. Only that you load
> it from a variable, instead of the function return type. So you could just
> v_ret instead of v_jumpaddr, and your code would be simpler and faster.
Ah, I see you mean to continue to use all the LLVMBuildCondBr()s as
the code currently does, but use v_ret like in the code above, instead
of v_jumpaddr, to access the jump address returned by the
step-choosing function. I've done that in the updated patch. This
also allows us to get rid of all the jumpdone fields in the
ExprEvalStep.
I've also moved the blocks of code in ExecInitExprRec() that
initialize the state for JsonExpr and JsonItemCoercions into new
functions. I've also moved EvalJsonPathVar() from execExprInterp.c to
jsonpath_exec.c where it's used.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
v6-0001-Some-improvements-to-JsonExpr-evaluation.patch (application/octet-stream, 58.7K) download
From 87956c96df4902eaacbaee4a810bc0199135d76f Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Wed, 22 Jun 2022 12:00:47 -0400
Subject: [PATCH v6] Some improvements to JsonExpr evaluation
* Store a pointer to the returning type input function FmgrInfo,
not the struct itself, in JsonExprState
* Evaluate various JsonExpr sub-expressions using parent ExprState
These include PASSING args, ON ERROR, ON EMPTY default expressions.
PASSING args are simple, because they all must be evaluated before
the JSON path evaluation can occur anyway, so pushing those
expressions to be uncondionally evaluated before the step to
evaluate JsonExpr proper suffices. Evaluation of the ON ERROR and
the ON EMPTY expressions is conditional on what JSON item pops out
of the path evaluation (ExecEvalJson()), so needs more logic to
gate the steps for their expressions that are pushed into the parent
ExprState, rather than be imlemented using separate ExprStates.
To that end, this moves the ON ERROR, ON EMPTY behavior decision logic
out of ExecEvalJson() and ExecEvalJsonExpr() into a new function
ExecEvalJsonExprBehavior().
* Final result coercion is now also computed as a separate step that
gets pushed into the parent ExprState. Though given that any
coercion evaluation errors must be handled the same way as the errors
of JSON item evaluation, that is, to be handled according the ON ERROR
behavior specification, a sub-transaction must be used around coercion
evaluation, so the ExprState to use for the same must have to be a
different one from the JsonExpr's. However, instead of using one
ExprState for each JsonCoercion member of JsonItemCoercions, only one
is used for the whole JsonItemCoercions. The individual JsonCoercion
members (or their Exprs) are handled by steps pushed into that
ExprState, of which only the one that applies to a given JSON item is
chosen at the runtime and others "jumped" over.
* Move the logic of deciding whether to use a sub-transaction for JSON
path evaluation and subsequent coercion to ExecInitExprRec() instead
of recomputing the same thing on every evaluation.
---
src/backend/executor/execExpr.c | 447 +++++++++++++++++++-----
src/backend/executor/execExprInterp.c | 474 +++++++++++++++++---------
src/backend/jit/llvm/llvmjit_expr.c | 220 +++++++++++-
src/backend/jit/llvm/llvmjit_types.c | 4 +
src/backend/optimizer/util/clauses.c | 6 +-
src/backend/utils/adt/jsonpath_exec.c | 68 ++++
src/include/executor/execExpr.h | 193 ++++++++---
src/include/utils/jsonb.h | 2 +
src/include/utils/jsonpath.h | 3 -
9 files changed, 1121 insertions(+), 296 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index d0a57c7aae..37d2acbc8b 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -86,6 +86,10 @@ static void ExecBuildAggTransCall(ExprState *state, AggState *aggstate,
FunctionCallInfo fcinfo, AggStatePerTrans pertrans,
int transno, int setno, int setoff, bool ishash,
bool nullcheck);
+static void ExecInitJsonExpr(ExprEvalStep *scratch, JsonExpr *jexpr, ExprState *state,
+ Datum *resv, bool *resnull);
+static void ExecInitJsonItemCoercions(ExprEvalStep *scratch, JsonItemCoercions *coercions,
+ ExprState *state, Datum *resv, bool *resnull);
static ExprState *
@@ -178,7 +182,7 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
* ExecInitExprWithCaseValue: prepare an expression tree for execution
*
* This is the same as ExecInitExpr, except that a pointer to the value for
- * CasTestExpr is passed here.
+ * CaseTestExpr is passed here.
*/
ExprState *
ExecInitExprWithCaseValue(Expr *node, PlanState *parent,
@@ -2561,110 +2565,54 @@ ExecInitExprRec(Expr *node, ExprState *state,
case T_JsonExpr:
{
JsonExpr *jexpr = castNode(JsonExpr, node);
- JsonExprState *jsestate = palloc0(sizeof(JsonExprState));
- ListCell *argexprlc;
- ListCell *argnamelc;
- scratch.opcode = EEOP_JSONEXPR;
- scratch.d.jsonexpr.jsestate = jsestate;
-
- jsestate->jsexpr = jexpr;
-
- jsestate->formatted_expr =
- palloc(sizeof(*jsestate->formatted_expr));
-
- ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
- &jsestate->formatted_expr->value,
- &jsestate->formatted_expr->isnull);
-
- jsestate->pathspec =
- palloc(sizeof(*jsestate->pathspec));
-
- ExecInitExprRec((Expr *) jexpr->path_spec, state,
- &jsestate->pathspec->value,
- &jsestate->pathspec->isnull);
-
- jsestate->res_expr =
- palloc(sizeof(*jsestate->res_expr));
-
- jsestate->result_expr = jexpr->result_coercion
- ? ExecInitExprWithCaseValue((Expr *) jexpr->result_coercion->expr,
- state->parent,
- &jsestate->res_expr->value,
- &jsestate->res_expr->isnull)
- : NULL;
-
- jsestate->default_on_empty = !jexpr->on_empty ? NULL :
- ExecInitExpr((Expr *) jexpr->on_empty->default_expr,
- state->parent);
-
- jsestate->default_on_error =
- ExecInitExpr((Expr *) jexpr->on_error->default_expr,
- state->parent);
+ ExecInitJsonExpr(&scratch, jexpr, state, resv, resnull);
+ break;
+ }
- if (jexpr->omit_quotes ||
- (jexpr->result_coercion && jexpr->result_coercion->via_io))
- {
- Oid typinput;
+ case T_JsonItemCoercions:
+ {
+ JsonItemCoercions *coercions = castNode(JsonItemCoercions,
+ node);
- /* lookup the result type's input function */
- getTypeInputInfo(jexpr->returning->typid, &typinput,
- &jsestate->input.typioparam);
- fmgr_info(typinput, &jsestate->input.func);
- }
+ ExecInitJsonItemCoercions(&scratch, coercions, state, resv,
+ resnull);
+ break;
+ }
- jsestate->args = NIL;
+ case T_JsonCoercion:
+ {
+ JsonCoercion *coercion = castNode(JsonCoercion, node);
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
- forboth(argexprlc, jexpr->passing_values,
- argnamelc, jexpr->passing_names)
- {
- Expr *argexpr = (Expr *) lfirst(argexprlc);
- String *argname = lfirst_node(String, argnamelc);
- JsonPathVariableEvalContext *var = palloc(sizeof(*var));
-
- var->name = pstrdup(argname->sval);
- var->typid = exprType((Node *) argexpr);
- var->typmod = exprTypmod((Node *) argexpr);
- var->estate = ExecInitExpr(argexpr, state->parent);
- var->econtext = NULL;
- var->mcxt = NULL;
- var->evaluated = false;
- var->value = (Datum) 0;
- var->isnull = true;
-
- jsestate->args =
- lappend(jsestate->args, var);
- }
+ /*
+ * Only ever get here for JsonExpr.result_expression and the
+ * caller should have checked this.
+ */
+ Assert(coercion->expr);
- jsestate->cache = NULL;
+ /*
+ * Push a step to read the value provided by the parent
+ * JsonExpr via a CaseTestExpr.
+ */
+ scratch.opcode = EEOP_CASE_TESTVAL;
+ scratch.d.casetest.value = state->innermost_caseval;
+ scratch.d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, &scratch);
- if (jexpr->coercions)
- {
- JsonCoercion **coercion;
- struct JsonCoercionState *cstate;
- Datum *caseval;
- bool *casenull;
+ /* Push step(s) to compute coercion->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
- jsestate->coercion_expr =
- palloc(sizeof(*jsestate->coercion_expr));
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
- caseval = &jsestate->coercion_expr->value;
- casenull = &jsestate->coercion_expr->isnull;
+ ExecInitExprRec((Expr *) coercion->expr, state, resv, resnull);
- for (cstate = &jsestate->coercions.null,
- coercion = &jexpr->coercions->null;
- coercion <= &jexpr->coercions->composite;
- coercion++, cstate++)
- {
- cstate->coercion = *coercion;
- cstate->estate = *coercion ?
- ExecInitExprWithCaseValue((Expr *) (*coercion)->expr,
- state->parent,
- caseval, casenull) : NULL;
- }
- }
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
- ExprEvalPushStep(state, &scratch);
break;
}
@@ -4257,3 +4205,314 @@ ExecBuildParamSetEqual(TupleDesc desc,
return state;
}
+
+/*
+ * Push steps to evaluate a JsonExpr and its various subsidiary expressions.
+ */
+static void
+ExecInitJsonExpr(ExprEvalStep *scratch, JsonExpr *jexpr, ExprState *state,
+ Datum *resv, bool *resnull)
+{
+ JsonExprState *jsestate = palloc0(sizeof(JsonExprState));
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ ListCell *argexprlc;
+ ListCell *argnamelc;
+ int skip_step_off;
+ int passing_args_step_off;
+ int behavior_step_off;
+ int onempty_default_step_off;
+ int onerror_default_step_off;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
+ int coercion_step_off;
+ ExprEvalStep *as;
+ bool throwErrors = (jexpr->on_error->btype == JSON_BEHAVIOR_ERROR);
+
+ jsestate->jsexpr = jexpr;
+
+ /*
+ * Add steps to compute formatted_expr, pathspec, and PASSING arg
+ * expressions as things that must be evaluated *before* the actual JSON
+ * path expression.
+ */
+ ExecInitExprRec((Expr *) jexpr->formatted_expr, state,
+ &pre_eval->formatted_expr.value,
+ &pre_eval->formatted_expr.isnull);
+ ExecInitExprRec((Expr *) jexpr->path_spec, state,
+ &pre_eval->pathspec.value,
+ &pre_eval->pathspec.isnull);
+
+ /*
+ * Before pushing steps for PASSING args, push a step to decide whether to
+ * skip evaluating the args and the JSON path expression depending on
+ * whether either of formatted_expr and pathspec is NULL.
+ */
+ scratch->opcode = EEOP_JSONEXPR_SKIP;
+ scratch->d.jsonexpr_skip.jsestate = jsestate;
+ skip_step_off = state->steps_len;
+ ExprEvalPushStep(state, scratch);
+
+ /* PASSING args. */
+ jsestate->pre_eval.args = NIL;
+ passing_args_step_off = state->steps_len;
+ forboth(argexprlc, jexpr->passing_values,
+ argnamelc, jexpr->passing_names)
+ {
+ Expr *argexpr = (Expr *) lfirst(argexprlc);
+ String *argname = lfirst_node(String, argnamelc);
+ JsonPathVariableEvalContext *var = palloc(sizeof(*var));
+
+ var->name = pstrdup(argname->sval);
+ var->typid = exprType((Node *) argexpr);
+ var->typmod = exprTypmod((Node *) argexpr);
+
+ /*
+ * A separate ExprState is not necessary for these expressions when
+ * being evaluated for a JsonExpr, like in this case, because they
+ * will evaluated as the steps of the JsonExpr.
+ */
+ var->estate = NULL;
+ var->econtext = NULL;
+ var->mcxt = NULL;
+
+ /*
+ * Mark these as always evaluated because they must have been evaluated
+ * before JSON path evaluation begins, because we haven't pushed the
+ * step for the latter yet.
+ */
+ var->evaluated = true;
+
+ ExecInitExprRec((Expr *) argexpr, state, &var->value, &var->isnull);
+
+ pre_eval->args = lappend(pre_eval->args, var);
+ }
+
+ /* Step for the actual JSON path evaluation. */
+ scratch->opcode = EEOP_JSONEXPR_PATH;
+ scratch->d.jsonexpr.jsestate = jsestate;
+ ExprEvalPushStep(state, scratch);
+
+ /*
+ * Push steps to control the evaluation of expressions based
+ * on the result of JSON path evaluation.
+ */
+
+ /* Step to handle ON ERROR and ON EMPTY behavior */
+ scratch->opcode = EEOP_JSONEXPR_BEHAVIOR;
+ scratch->d.jsonexpr_behavior.jsestate = jsestate;
+ behavior_step_off = state->steps_len;
+ ExprEvalPushStep(state, scratch);
+
+ /* Step(s) to evaluate ON EMPTY default expression */
+ onempty_default_step_off = state->steps_len;
+ if (jexpr->on_empty && jexpr->on_empty->default_expr)
+ {
+ ExecInitExprRec((Expr *) jexpr->on_empty->default_expr,
+ state, resv, resnull);
+
+ /*
+ * Emit JUMP step to jump to end of JsonExpr code, because evaluating
+ * the default expression gives the final result and there's nothing
+ * more to do.
+ */
+ scratch->opcode = EEOP_JUMP;
+ scratch->d.jump.jumpdone = -1; /* computed later */
+ ExprEvalPushStep(state, scratch);
+
+ /*
+ * Don't know address for that jump yet, compute once the whole
+ * JsonExpr is built.
+ */
+ adjust_jumps = lappend_int(adjust_jumps, state->steps_len - 1);
+ }
+
+ /* Step(s) to evaluate ON ERROR default expression */
+ onerror_default_step_off = state->steps_len;
+ if (jexpr->on_error && jexpr->on_error->default_expr)
+ {
+ ExecInitExprRec((Expr *) jexpr->on_error->default_expr,
+ state, resv, resnull);
+
+ /* See the comment above. */
+ scratch->opcode = EEOP_JUMP;
+ scratch->d.jump.jumpdone = -1; /* computed later */
+ ExprEvalPushStep(state, scratch);
+ adjust_jumps = lappend_int(adjust_jumps, state->steps_len - 1);
+ }
+
+ /*
+ * Step to handle applying coercion to the JSON item returned by
+ * EEOP_JSONEXPR_PATH or to the ON EMPTY/ERROR expression as
+ * EEOP_JSONEXPR_BEHAVIOR decides.
+ */
+ scratch->opcode = EEOP_JSONEXPR_COERCION;
+ scratch->d.jsonexpr_coercion.jsestate = jsestate;
+ coercion_step_off = state->steps_len;
+ ExprEvalPushStep(state, scratch);
+
+ /*
+ * Adjust jump target addresses in various post-eval steps now that we have
+ * all the steps in place.
+ */
+
+ /* EEOP_JSONEXPR_SKIP */
+ as = &state->steps[skip_step_off];
+ as->d.jsonexpr_skip.jump_coercion = coercion_step_off;
+ as->d.jsonexpr_skip.jump_passing_args = passing_args_step_off;
+
+ /* EEOP_JSONEXPR_BEHAVIOR */
+ as = &state->steps[behavior_step_off];
+ as->d.jsonexpr_behavior.jump_onerror_default = onerror_default_step_off;
+ as->d.jsonexpr_behavior.jump_onempty_default = onempty_default_step_off;
+ as->d.jsonexpr_behavior.jump_coercion = coercion_step_off;
+ as->d.jsonexpr_behavior.jump_skip_coercion = coercion_step_off + 1;
+
+ /* EEOP_JSONEXPR_COERCION */
+ as = &state->steps[coercion_step_off];
+ as->d.jsonexpr_coercion.jump_coercion_error = behavior_step_off;
+ as->d.jsonexpr_coercion.jump_coercion_done = coercion_step_off + 1;
+
+ /*
+ * EEOP_JUMP steps added after ON EMPTY and ON ERROR default expression
+ * should jump to the current step address.
+ */
+ foreach(lc, adjust_jumps)
+ {
+ as = &state->steps[lfirst_int(lc)];
+ as->d.jump.jumpdone = state->steps_len;
+ }
+
+ /*
+ * Set if we must use a sub-transaction around path evaluation
+ * that can be aborted on error instead of aborting the parent
+ * query.
+ */
+ jsestate->needSubtrans = ExecEvalJsonNeedsSubTransaction(jexpr, true);
+
+ /*
+ * Set if coercion, which runs separately from path evaluation and whose
+ * errors must be caught and handled per ON ERROR behavior, must use a
+ * sub-transaction.
+ */
+ jsestate->coercionNeedSubtrans = !throwErrors;
+
+ /*
+ * Set RETURNING type's input function used by ExecEvalJsonExprCoercion().
+ */
+ if (jexpr->omit_quotes ||
+ (jexpr->result_coercion && jexpr->result_coercion->via_io))
+ {
+ Oid typinput;
+ FmgrInfo *finfo;
+
+ /* lookup the result type's input function */
+ getTypeInputInfo(jexpr->returning->typid, &typinput,
+ &jsestate->input.typioparam);
+ finfo = palloc0(sizeof(FmgrInfo));
+ fmgr_info(typinput, finfo);
+ jsestate->input.finfo = finfo;
+ }
+
+ /*
+ * Initialize coercion expression(s). These can't be inlined into the
+ * parent ExprState, because of the need to use a sub-transaction around
+ * their evaluation.
+ */
+ if (jexpr->result_coercion && jexpr->result_coercion->expr)
+ jsestate->result_coercion =
+ ExecInitExprWithCaseValue((Expr *) jexpr->result_coercion,
+ state->parent, resv, resnull);
+ if (jexpr->coercions)
+ jsestate->coercions =
+ ExecInitExprWithCaseValue((Expr *) jexpr->coercions,
+ state->parent, resv, resnull);
+}
+
+/*
+ * Push steps to evaluate a JsonItemCoercions, which contains the state for
+ * evaluating all possible coercions that may need to be applied to a JSON
+ * item coming from evaluating the parent JsonExpr.
+ */
+static void
+ExecInitJsonItemCoercions(ExprEvalStep *scratch, JsonItemCoercions *coercions,
+ ExprState *state, Datum *resv, bool *resnull)
+{
+ JsonItemCoercionsState *jcstate = palloc0(sizeof(JsonItemCoercionsState));
+ JsonCoercion **coercion;
+ JsonItemCoercionState *cstate;
+ ExprEvalStep *as;
+ int json_item_coercion_step_id;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
+
+ /*
+ * Push a step to read the JSON item value passed by the parent JsonExpr
+ * via a CaseTestExpr.
+ */
+ scratch->opcode = EEOP_CASE_TESTVAL;
+ scratch->d.casetest.value = state->innermost_caseval;
+ scratch->d.casetest.isnull = state->innermost_casenull;
+ ExprEvalPushStep(state, scratch);
+
+ /* Push the control step. */
+ scratch->opcode = EEOP_JSON_ITEM_COERCION;
+ scratch->d.jsonexpr_item_coercion.jcstate = jcstate;
+ json_item_coercion_step_id = state->steps_len;
+ ExprEvalPushStep(state, scratch);
+ /* Will set jump_skip_coercion target address below. */
+
+ /* Now push the steps of individual coercion's expression, if needed. */
+ for (cstate = &jcstate->null,
+ coercion = &coercions->null;
+ coercion <= &coercions->composite;
+ coercion++, cstate++)
+ {
+ cstate->coercion = *coercion;
+ if (cstate->coercion && cstate->coercion->expr)
+ {
+ Datum *save_innermost_caseval;
+ bool *save_innermost_isnull;
+
+ cstate->jump_eval_expr = state->steps_len;
+
+ /* Push step(s) to compute (*coercion)->expr. */
+ save_innermost_caseval = state->innermost_caseval;
+ save_innermost_isnull = state->innermost_casenull;
+
+ state->innermost_caseval = resv;
+ state->innermost_casenull = resnull;
+
+ ExecInitExprRec((Expr *) cstate->coercion->expr,
+ state, resv, resnull);
+
+ state->innermost_caseval = save_innermost_caseval;
+ state->innermost_casenull = save_innermost_isnull;
+ }
+ else
+ cstate->jump_eval_expr = -1;
+
+ /* Emit JUMP step to skip past other coercions' steps. */
+ scratch->opcode = EEOP_JUMP;
+
+ /*
+ * Remember JUMP step address to set the actual jump target addreess
+ * below.
+ */
+ adjust_jumps = lappend_int(adjust_jumps, state->steps_len);
+ ExprEvalPushStep(state, scratch);
+ }
+
+ /*
+ * Adjust the jump target address in the control step and all
+ * the jumps we added in the above loop to jump here.
+ */
+ as = &state->steps[json_item_coercion_step_id];
+ as->d.jsonexpr_item_coercion.jump_skip_item_coercion = state->steps_len;
+ foreach(lc, adjust_jumps)
+ {
+ int jump_step_id = lfirst_int(lc);
+
+ as = &state->steps[jump_step_id];
+ as->d.jump.jumpdone = state->steps_len;
+ }
+}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 636794ca6f..6648c2b9b8 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -158,6 +158,15 @@ static TupleDesc get_cached_rowtype(Oid type_id, int32 typmod,
bool *changed);
static void ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, bool checkisnull);
+static Datum ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null);
+typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
+ Datum item, bool *resnull, void *p, bool *error);
+static Datum ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
+ ExprContext *econtext,
+ Datum res, bool *resnull,
+ void *p, bool *error, bool subtrans);
+static Datum ExecEvalJsonCoercion(ExprEvalStep *op, ExprContext *econtext,
+ Datum res, bool *isNull, void *p, bool *error);
/* fast-path evaluation functions */
static Datum ExecJustInnerVar(ExprState *state, ExprContext *econtext, bool *isnull);
@@ -490,7 +499,11 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SUBPLAN,
&&CASE_EEOP_JSON_CONSTRUCTOR,
&&CASE_EEOP_IS_JSON,
- &&CASE_EEOP_JSONEXPR,
+ &&CASE_EEOP_JSONEXPR_SKIP,
+ &&CASE_EEOP_JSONEXPR_PATH,
+ &&CASE_EEOP_JSONEXPR_BEHAVIOR,
+ &&CASE_EEOP_JSONEXPR_COERCION,
+ &&CASE_EEOP_JSON_ITEM_COERCION,
&&CASE_EEOP_AGG_STRICT_DESERIALIZE,
&&CASE_EEOP_AGG_DESERIALIZE,
&&CASE_EEOP_AGG_STRICT_INPUT_CHECK_ARGS,
@@ -1841,13 +1854,37 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
- EEO_CASE(EEOP_JSONEXPR)
+ EEO_CASE(EEOP_JSONEXPR_PATH)
{
/* too complex for an inline implementation */
ExecEvalJson(state, op, econtext);
EEO_NEXT();
}
+ EEO_CASE(EEOP_JSONEXPR_SKIP)
+ {
+ /* too complex for an inline implementation */
+ EEO_JUMP(ExecEvalJsonExprSkip(state, op));
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_BEHAVIOR)
+ {
+ /* too complex for an inline implementation */
+ EEO_JUMP(ExecEvalJsonExprBehavior(state, op));
+ }
+
+ EEO_CASE(EEOP_JSONEXPR_COERCION)
+ {
+ /* too complex for an inline implementation */
+ EEO_JUMP(ExecEvalJsonExprCoercion(state, op, econtext));
+ }
+
+ EEO_CASE(EEOP_JSON_ITEM_COERCION)
+ {
+ /* too complex for an inline implementation */
+ EEO_JUMP(ExecEvalJsonExprItemCoercion(state, op));
+ }
+
EEO_CASE(EEOP_LAST)
{
/* unreachable */
@@ -4704,8 +4741,7 @@ ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
* Evaluate a JSON error/empty behavior result.
*/
static Datum
-ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
- ExprState *default_estate, bool *is_null)
+ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null)
{
*is_null = false;
@@ -4730,7 +4766,9 @@ ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
return (Datum) 0;
case JSON_BEHAVIOR_DEFAULT:
- return ExecEvalExpr(default_estate, econtext, is_null);
+ /* Always handled in the caller. */
+ Assert(false);
+ return (Datum) 0;
default:
elog(ERROR, "unrecognized SQL/JSON behavior %d", behavior->btype);
@@ -4742,17 +4780,18 @@ ExecEvalJsonBehavior(ExprContext *econtext, JsonBehavior *behavior,
* Evaluate a coercion of a JSON item to the target type.
*/
static Datum
-ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
- Datum res, bool *isNull, void *p, bool *error)
+ExecEvalJsonCoercion(ExprEvalStep *op, ExprContext *econtext,
+ Datum res, bool *isNull, void *p, bool *error)
{
- ExprState *estate = p;
- JsonExprState *jsestate;
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ ExprState *estate = post_eval->coercion;
+
+ post_eval->coercion_error = false;
if (estate) /* coerce using specified expression */
return ExecEvalExpr(estate, econtext, isNull);
- jsestate = op->d.jsonexpr.jsestate;
-
if (jsestate->jsexpr->op != JSON_EXISTS_OP)
{
JsonCoercion *coercion = jsestate->jsexpr->result_coercion;
@@ -4766,7 +4805,7 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
/* strip quotes and call typinput function */
char *str = *isNull ? NULL : JsonbUnquote(jb);
- return InputFunctionCall(&jsestate->input.func, str,
+ return InputFunctionCall(jsestate->input.finfo, str,
jsestate->input.typioparam,
jexpr->returning->typmod);
}
@@ -4774,73 +4813,85 @@ ExecEvalJsonExprCoercion(ExprEvalStep *op, ExprContext *econtext,
return json_populate_type(res, JSONBOID,
jexpr->returning->typid,
jexpr->returning->typmod,
- &jsestate->cache,
+ &post_eval->cache,
econtext->ecxt_per_query_memory,
isNull);
}
- if (jsestate->result_expr)
+ /* Coerce with jexpr->result_coercion if there's one */
+ if (jsestate->result_coercion)
{
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *isNull;
-
- res = ExecEvalExpr(jsestate->result_expr, econtext, isNull);
+ *op->resvalue = res;
+ *op->resnull = *isNull;
+ return ExecEvalExpr(jsestate->result_coercion, econtext, isNull);
}
return res;
}
/*
- * Evaluate a JSON path variable caching computed value.
+ * Return a coercion among those given in 'coercions' for given
+ * JSON item.
*/
-int
-EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
- JsonbValue *val, JsonbValue *baseObject)
+JsonCoercion *
+ExecGetJsonItemCoercion(JsonbValue *item, JsonItemCoercions *coercions)
{
- JsonPathVariableEvalContext *var = NULL;
- List *vars = cxt;
- ListCell *lc;
- int id = 1;
+ if (item->type == jbvBinary &&
+ JsonContainerIsScalar(item->val.binary.data))
+ {
+ JsonbValue buf;
+ bool res PG_USED_FOR_ASSERTS_ONLY;
- if (!varName)
- return list_length(vars);
+ res = JsonbExtractScalar(item->val.binary.data, &buf);
+ item = &buf;
+ Assert(res);
+ }
- foreach(lc, vars)
+ /* get coercion state reference and datum of the corresponding SQL type */
+ switch (item->type)
{
- var = lfirst(lc);
-
- if (!strncmp(var->name, varName, varNameLen))
- break;
+ case jbvNull:
+ return coercions->null;
- var = NULL;
- id++;
- }
+ case jbvString:
+ return coercions->string;
- if (!var)
- return -1;
+ case jbvNumeric:
+ return coercions->numeric;
- if (!var->evaluated)
- {
- MemoryContext oldcxt = var->mcxt ?
- MemoryContextSwitchTo(var->mcxt) : NULL;
+ case jbvBool:
+ return coercions->boolean;
- var->value = ExecEvalExpr(var->estate, var->econtext, &var->isnull);
- var->evaluated = true;
+ case jbvDatetime:
+ switch (item->val.datetime.typid)
+ {
+ case DATEOID:
+ return coercions->date;
+ case TIMEOID:
+ return coercions->time;
+ case TIMETZOID:
+ return coercions->timetz;
+ case TIMESTAMPOID:
+ return coercions->timestamp;
+ case TIMESTAMPTZOID:
+ return coercions->timestamptz;
+ default:
+ elog(ERROR, "unexpected jsonb datetime type oid %u",
+ item->val.datetime.typid);
+ }
+ break;
- if (oldcxt)
- MemoryContextSwitchTo(oldcxt);
- }
+ case jbvArray:
+ case jbvObject:
+ case jbvBinary:
+ return coercions->composite;
- if (var->isnull)
- {
- val->type = jbvNull;
- return 0;
+ default:
+ elog(ERROR, "unexpected jsonb value type %d", item->type);
}
- JsonItemFromDatum(var->value, var->typid, var->typmod, val);
-
- *baseObject = *val;
- return id;
+ Assert(false);
+ return NULL;
}
/*
@@ -4848,14 +4899,14 @@ EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
* corresponding SQL type and a pointer to the coercion state.
*/
Datum
-ExecPrepareJsonItemCoercion(JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pcoercion)
+ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pcoercion)
{
- struct JsonCoercionState *coercion;
+ JsonbValue *item = DatumGetJsonbValueP(itemval);
+ JsonItemCoercionState *coercion;
Datum res;
- JsonbValue buf;
+ JsonbValue buf;
if (item->type == jbvBinary &&
JsonContainerIsScalar(item->val.binary.data))
@@ -4934,9 +4985,6 @@ ExecPrepareJsonItemCoercion(JsonbValue *item,
return res;
}
-typedef Datum (*JsonFunc) (ExprEvalStep *op, ExprContext *econtext,
- Datum item, bool *resnull, void *p, bool *error);
-
static Datum
ExecEvalJsonExprSubtrans(JsonFunc func, ExprEvalStep *op,
ExprContext *econtext,
@@ -5007,7 +5055,6 @@ typedef struct
{
JsonPath *path;
bool *error;
- bool coercionInSubtrans;
} ExecEvalJsonExprContext;
static Datum
@@ -5018,16 +5065,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
ExecEvalJsonExprContext *cxt = pcxt;
JsonPath *path = cxt->path;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
- ExprState *estate = NULL;
- bool empty = false;
+ bool *empty = &post_eval->empty;
Datum res = (Datum) 0;
switch (jexpr->op)
{
case JSON_QUERY_OP:
- res = JsonPathQuery(item, path, jexpr->wrapper, &empty, error,
- jsestate->args);
+ res = JsonPathQuery(item, path, jexpr->wrapper, empty, error,
+ pre_eval->args);
if (error && *error)
{
*resnull = true;
@@ -5038,17 +5086,20 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
case JSON_VALUE_OP:
{
- struct JsonCoercionState *jcstate;
- JsonbValue *jbv = JsonPathValue(item, path, &empty, error,
- jsestate->args);
+ JsonCoercion *coercion;
+ JsonbValue *jbv = JsonPathValue(item, path, empty, error,
+ pre_eval->args);
if (error && *error)
+ {
+ *resnull = true;
return (Datum) 0;
+ }
if (!jbv) /* NULL or empty */
break;
- Assert(!empty);
+ Assert(!*empty);
*resnull = false;
@@ -5061,19 +5112,17 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
break;
}
- /* Use coercion from SQL/JSON item type to the output type */
- res = ExecPrepareJsonItemCoercion(jbv,
- jsestate->jsexpr->returning,
- &jsestate->coercions,
- &jcstate);
-
- if (jcstate->coercion &&
- (jcstate->coercion->via_io ||
- jcstate->coercion->via_populate))
+ /*
+ * Error out if no cast exists to coerce SQL/JSON item to the
+ * the output type
+ */
+ coercion = ExecGetJsonItemCoercion(jbv, jsestate->jsexpr->coercions);
+ if (coercion && (coercion->via_io || coercion->via_populate))
{
if (error)
{
*error = true;
+ *resnull = true;
return (Datum) 0;
}
@@ -5085,32 +5134,42 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_SQL_JSON_ITEM_CANNOT_BE_CAST_TO_TARGET_TYPE),
errmsg("SQL/JSON item cannot be cast to target type")));
}
- else if (!jcstate->estate)
- return res; /* no coercion */
-
- /* coerce using specific expression */
- estate = jcstate->estate;
- jsestate->coercion_expr->value = res;
- jsestate->coercion_expr->isnull = *resnull;
+ else
+ {
+ /*
+ * Coerce using a specific coercion (JsonItemCoercion).
+ * Note we are passing the JsonbValue as is, without
+ * converting it to Jsonb, because
+ * ExecPrepareJsonItemCoercion() expects the JSON item
+ * to be in that format.
+ */
+ res = JsonbValuePGetDatum(jbv);
+ post_eval->coercion = jsestate->coercions;
+ }
break;
}
case JSON_EXISTS_OP:
{
bool exists = JsonPathExists(item, path,
- jsestate->args,
+ pre_eval->args,
error);
*resnull = error && *error;
res = BoolGetDatum(exists);
- if (!jsestate->result_expr)
+ if (jexpr->result_coercion == NULL)
+ {
+ /* No coercion needed */
+ post_eval->coercion_done = true;
return res;
-
- /* coerce using result expression */
- estate = jsestate->result_expr;
- jsestate->res_expr->value = res;
- jsestate->res_expr->isnull = *resnull;
+ }
+ else
+ {
+ /* coerce using result expression */
+ Assert(jsestate->result_coercion);
+ post_eval->coercion = jsestate->result_coercion;
+ }
break;
}
@@ -5123,7 +5182,11 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
return (Datum) 0;
}
- if (empty)
+ /*
+ * If the ON EMPTY behavior is to cause an error, do so here. Other
+ * behaviors will be handled by the caller.
+ */
+ if (*empty)
{
Assert(jexpr->on_empty); /* it is not JSON_EXISTS */
@@ -5139,29 +5202,13 @@ ExecEvalJsonExpr(ExprEvalStep *op, ExprContext *econtext,
(errcode(ERRCODE_NO_SQL_JSON_ITEM),
errmsg("no SQL/JSON item")));
}
-
- if (jexpr->on_empty->btype == JSON_BEHAVIOR_DEFAULT)
-
- /*
- * Execute DEFAULT expression as a coercion expression, because
- * its result is already coerced to the target type.
- */
- estate = jsestate->default_on_empty;
- else
- /* Execute ON EMPTY behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_empty,
- jsestate->default_on_empty,
- resnull);
}
- return ExecEvalJsonExprSubtrans(ExecEvalJsonExprCoercion, op, econtext,
- res, resnull, estate, error,
- cxt->coercionInSubtrans);
+ return res;
}
bool
-ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *coercions)
+ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool consider_coerce)
{
if (jsexpr->on_error->btype == JSON_BEHAVIOR_ERROR)
return false;
@@ -5169,12 +5216,155 @@ ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
if (jsexpr->op == JSON_EXISTS_OP && !jsexpr->result_coercion)
return false;
- if (!coercions)
+ if (!consider_coerce)
return true;
return false;
}
+/* Skip calling ExecEvalJson() on a JsonExpr? */
+int
+ExecEvalJsonExprSkip(ExprState *state, ExprEvalStep *op)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_skip.jsestate;
+
+ /*
+ * Skip if either of the input expressions has turned out to be NULL,
+ * though do execute domain checks for NULLs, which are handled by the
+ * coercion step.
+ */
+ if (jsestate->pre_eval.formatted_expr.isnull ||
+ jsestate->pre_eval.pathspec.isnull)
+ {
+ *op->resvalue = 0;
+ *op->resnull = true;
+ Assert(jsestate->post_eval.coercion == NULL);
+ /* A signal to the coercion step to not use a sub-trancaction. */
+ jsestate->post_eval.coercion_error = true;
+ return op->d.jsonexpr_skip.jump_coercion;
+ }
+
+ /*
+ * Go evaluate the PASSING args if any and subsequently JSON path
+ * itself.
+ */
+ return op->d.jsonexpr_skip.jump_passing_args;
+}
+
+int
+ExecEvalJsonExprBehavior(ExprState *state, ExprEvalStep *op)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_behavior.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+ JsonBehavior *behavior = NULL;
+ int jump_to;
+ bool error = (post_eval->error || post_eval->coercion_error);
+
+ /*
+ * Set if the coercion step, if will run afterwards, should use a sub-
+ * transaction to be aborted on error instead of throwing the error.
+ * Don't use a sub-transaction if coercing the ON ERROR expression,
+ * though.
+ *
+ * XXX - doesn't that violate ON ERROR behavior? Actually, that is
+ * how it behaved even before this expression eval logic rewrite!
+ */
+ post_eval->coercion_use_subtrans =
+ (jsestate->coercionNeedSubtrans && !error);
+
+ /*
+ * If no error or the JSON item is not empty, directly go to the coercion
+ * step to coerce the item as is, or skip the coercion step if the item is
+ * already coerced by ExecEvalJson().
+ */
+ if (!error && !post_eval->empty)
+ return !post_eval->coercion_done ?
+ op->d.jsonexpr_behavior.jump_coercion :
+ op->d.jsonexpr_behavior.jump_skip_coercion;
+
+ if (error)
+ {
+ behavior = jsestate->jsexpr->on_error;
+ jump_to = op->d.jsonexpr_behavior.jump_onerror_default;
+ }
+ else if (post_eval->empty)
+ {
+ behavior = jsestate->jsexpr->on_empty;
+ jump_to = op->d.jsonexpr_behavior.jump_onempty_default;
+ }
+
+ Assert(behavior);
+
+ /*
+ * If a non-default behavior is specified, get the appropriate value and go
+ * to the coercion step.
+ */
+ if (behavior->btype != JSON_BEHAVIOR_DEFAULT)
+ {
+ *op->resvalue = ExecEvalJsonBehavior(behavior, op->resnull);
+
+ if (post_eval->coercion_error)
+ {
+ post_eval->coercion_done = false;
+ post_eval->coercion = NULL;
+ }
+
+ jump_to = op->d.jsonexpr_behavior.jump_coercion;
+ }
+
+ /*
+ * Else evaluate the default ON ERROR or ON EMPTY expression, with no
+ * coercion needed afterwards given that the expression is already
+ * coerced appropriately in the parser.
+ */
+
+ Assert(jump_to >= 0);
+ return jump_to;
+}
+
+/*
+ * Apply coercion to a JSON item using either JsonExpr.result_coercion or one
+ * of JsonExpr.coercions, possibly using a sub-transaction.
+ */
+int
+ExecEvalJsonExprCoercion(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
+{
+ JsonExprState *jsestate = op->d.jsonexpr_coercion.jsestate;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
+
+ *op->resvalue =
+ ExecEvalJsonExprSubtrans(ExecEvalJsonCoercion, op, econtext,
+ *op->resvalue, op->resnull, NULL,
+ &post_eval->coercion_error,
+ post_eval->coercion_use_subtrans);
+ if (post_eval->coercion_error)
+ return op->d.jsonexpr_coercion.jump_coercion_error;
+
+ return op->d.jsonexpr_coercion.jump_coercion_done;
+}
+
+/*
+ * Apply a coercion to a JSON item, choosing one among those present in
+ * JsonItemCoercions.
+ */
+int
+ExecEvalJsonExprItemCoercion(ExprState *state, ExprEvalStep *op)
+{
+ JsonItemCoercionsState *jcstate = op->d.jsonexpr_item_coercion.jcstate;
+ JsonItemCoercionState *cstate = NULL;
+
+ *op->resvalue =
+ ExecPrepareJsonItemCoercion(*op->resvalue, jcstate, &cstate);
+ if (cstate->coercion && cstate->coercion->expr)
+ {
+ Assert(cstate->jump_eval_expr >= 0);
+ return cstate->jump_eval_expr;
+ }
+
+ /* Skip over all of the steps added for this JsonItemCoercions. */
+ return op->d.jsonexpr_item_coercion.jump_skip_item_coercion;
+}
+
/* ----------------------------------------------------------------
* ExecEvalJson
* ----------------------------------------------------------------
@@ -5184,64 +5374,30 @@ ExecEvalJson(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
{
ExecEvalJsonExprContext cxt;
JsonExprState *jsestate = op->d.jsonexpr.jsestate;
+ JsonExprPreEvalState *pre_eval = &jsestate->pre_eval;
+ JsonExprPostEvalState *post_eval = &jsestate->post_eval;
JsonExpr *jexpr = jsestate->jsexpr;
Datum item;
Datum res = (Datum) 0;
JsonPath *path;
- ListCell *lc;
- bool error = false;
- bool needSubtrans;
bool throwErrors = jexpr->on_error->btype == JSON_BEHAVIOR_ERROR;
*op->resnull = true; /* until we get a result */
*op->resvalue = (Datum) 0;
- if (jsestate->formatted_expr->isnull || jsestate->pathspec->isnull)
- {
- /* execute domain checks for NULLs */
- (void) ExecEvalJsonExprCoercion(op, econtext, res, op->resnull,
- NULL, NULL);
+ item = pre_eval->formatted_expr.value;
+ path = DatumGetJsonPathP(pre_eval->pathspec.value);
- Assert(*op->resnull);
- return;
- }
-
- item = jsestate->formatted_expr->value;
- path = DatumGetJsonPathP(jsestate->pathspec->value);
-
- /* reset JSON path variable contexts */
- foreach(lc, jsestate->args)
- {
- JsonPathVariableEvalContext *var = lfirst(lc);
-
- var->econtext = econtext;
- var->evaluated = false;
- }
-
- needSubtrans = ExecEvalJsonNeedsSubTransaction(jexpr, &jsestate->coercions);
+ /* Reset JsonExprPostEvalState for this evaluation. */
+ memset(post_eval, 0, sizeof(*post_eval));
cxt.path = path;
- cxt.error = throwErrors ? NULL : &error;
- cxt.coercionInSubtrans = !needSubtrans && !throwErrors;
- Assert(!needSubtrans || cxt.error);
+ cxt.error = throwErrors ? NULL : &post_eval->error;
+ Assert(!jsestate->needSubtrans || cxt.error);
res = ExecEvalJsonExprSubtrans(ExecEvalJsonExpr, op, econtext, item,
op->resnull, &cxt, cxt.error,
- needSubtrans);
-
- if (error)
- {
- /* Execute ON ERROR behavior */
- res = ExecEvalJsonBehavior(econtext, jexpr->on_error,
- jsestate->default_on_error,
- op->resnull);
-
- /* result is already coerced in DEFAULT behavior case */
- if (jexpr->on_error->btype != JSON_BEHAVIOR_DEFAULT)
- res = ExecEvalJsonExprCoercion(op, econtext, res,
- op->resnull,
- NULL, NULL);
- }
+ jsestate->needSubtrans);
*op->resvalue = res;
}
diff --git a/src/backend/jit/llvm/llvmjit_expr.c b/src/backend/jit/llvm/llvmjit_expr.c
index bd3965143d..2637f9c41d 100644
--- a/src/backend/jit/llvm/llvmjit_expr.c
+++ b/src/backend/jit/llvm/llvmjit_expr.c
@@ -2407,12 +2407,230 @@ llvm_compile_expr(ExprState *state)
LLVMBuildBr(b, opblocks[opno + 1]);
break;
- case EEOP_JSONEXPR:
+ case EEOP_JSONEXPR_PATH:
build_EvalXFunc(b, mod, "ExecEvalJson",
v_state, op, v_econtext);
LLVMBuildBr(b, opblocks[opno + 1]);
break;
+ case EEOP_JSONEXPR_SKIP:
+ {
+ LLVMValueRef params[2];
+ LLVMValueRef v_ret;
+
+ /*
+ * Call ExecEvalJsonExprSkip() to decide if JSON path
+ * evaluation can be skipped. This returns the step
+ * address to jump to.
+ */
+ params[0] = v_state;
+ params[1] = l_ptr_const(op, l_ptr(StructExprEvalStep));
+ v_ret = LLVMBuildCall(b,
+ llvm_pg_func(mod, "ExecEvalJsonExprSkip"),
+ params, lengthof(params), "");
+
+ /*
+ * Jump to coercion step if the returned address is the
+ * same as jsonexpr_skip.jump_coercion, which signifies
+ * skipping of JSON path evaluation, else to the next step
+ * which must point to the steps to evaluate PASSING args,
+ * if any, or to the JSON path evaluation.
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_ret,
+ l_int32_const(op->d.jsonexpr_skip.jump_coercion),
+ ""),
+ opblocks[op->d.jsonexpr_skip.jump_coercion],
+ opblocks[opno + 1]);
+ break;
+ }
+
+ case EEOP_JSONEXPR_BEHAVIOR:
+ {
+ LLVMValueRef params[2];
+ LLVMValueRef v_ret;
+ LLVMBasicBlockRef b_jump_skip_coercion,
+ b_jump_onerror_default;
+
+ /*
+ * Call ExecEvalJsonExprBehavior() to decide if ON EMPTY or
+ * ON ERROR behavior must be invoked depending on what JSON
+ * path evaluation returned. This returns the step address
+ * to jump to.
+ */
+ params[0] = v_state;
+ params[1] = l_ptr_const(op, l_ptr(StructExprEvalStep));
+ v_ret = LLVMBuildCall(b,
+ llvm_pg_func(mod, "ExecEvalJsonExprBehavior"),
+ params, lengthof(params), "");
+
+ b_jump_skip_coercion =
+ l_bb_before_v(opblocks[opno + 1],
+ "op.%d.jsonexpr_behavior_jump_skip_coercion", opno);
+ b_jump_onerror_default =
+ l_bb_before_v(opblocks[opno + 1],
+ "op.%d.jsonexpr_behavior_jump_onerror_default", opno);
+
+ /*
+ * Jump to coercion step if the returned address is the same as
+ * jsonexpr_behavior.jump_coercion, else to the block that checks
+ * whether to skip coercion.
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_ret,
+ l_int32_const(op->d.jsonexpr_behavior.jump_coercion),
+ ""),
+ opblocks[op->d.jsonexpr_behavior.jump_coercion],
+ b_jump_skip_coercion);
+
+ /*
+ * Block that checks whether to skip coercion.
+ *
+ * Jump to skip coercion if the returned address is the
+ * same as jsonexpr_behavior.jump_skip_coercion, else to
+ * the block that checks whether the ON ERROR or the ON
+ * EMPTY default expression must be evaluated instead.
+ */
+ LLVMPositionBuilderAtEnd(b, b_jump_skip_coercion);
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_ret,
+ l_int32_const(op->d.jsonexpr_behavior.jump_skip_coercion),
+ ""),
+ opblocks[op->d.jsonexpr_behavior.jump_skip_coercion],
+ b_jump_onerror_default);
+
+ /*
+ * Block that checks whether to evaluate the ON ERROR
+ * default expression.
+ *
+ * Jump to evaluate the ON ERROR default expression if the
+ * returned address is the same as
+ * jsonexpr_behavior.jump_onerror_default, else jump to
+ * evaluate the ON EMPTY default expression.
+ */
+ LLVMPositionBuilderAtEnd(b, b_jump_onerror_default);
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_ret,
+ l_int32_const(op->d.jsonexpr_behavior.jump_onerror_default),
+ ""),
+ opblocks[op->d.jsonexpr_behavior.jump_onerror_default],
+ opblocks[op->d.jsonexpr_behavior.jump_onempty_default]);
+ break;
+ }
+ case EEOP_JSONEXPR_COERCION:
+ {
+ LLVMValueRef params[3];
+ LLVMValueRef v_ret;
+
+ /*
+ * Call ExecEvalJsonExprCoercion() to evaluate appropriate
+ * coercion. This will return the step address to jump
+ * to.
+ */
+ params[0] = v_state;
+ params[1] = l_ptr_const(op, l_ptr(StructExprEvalStep));
+ params[2] = v_econtext;
+ v_ret = LLVMBuildCall(b,
+ llvm_pg_func(mod, "ExecEvalJsonExprCoercion"),
+ params, lengthof(params), "");
+
+ /*
+ * Jump to handle a coercion error if the returned address
+ * is the same as jsonexpr_coercion.jump_coercion_error,
+ * else to the step after coercion (coercion done!).
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_ret,
+ l_int32_const(op->d.jsonexpr_coercion.jump_coercion_error),
+ ""),
+ opblocks[op->d.jsonexpr_coercion.jump_coercion_error],
+ opblocks[op->d.jsonexpr_coercion.jump_coercion_done]);
+ break;
+ }
+ case EEOP_JSON_ITEM_COERCION:
+ {
+ JsonItemCoercionsState *jcstate = op->d.jsonexpr_item_coercion.jcstate;
+ JsonItemCoercionState *cstate;
+ LLVMValueRef params[2];
+ LLVMValueRef v_ret;
+ int n_coercions = (int) (&jcstate->composite - &jcstate->null) + 1;
+ int i;
+ LLVMBasicBlockRef *b_coercions;
+
+ /*
+ * Call ExecEvalJsonExprItemCoercion() to inspect the JSON
+ * item obtained by path evaluation and pick the coercion
+ * to apply accordingly. That will set
+ * jsonexpr_item_coercion.jumpdone to an address as
+ * described below.
+ */
+ params[0] = v_state;
+ params[1] = l_ptr_const(op, l_ptr(StructExprEvalStep));
+ v_ret = LLVMBuildCall(b,
+ llvm_pg_func(mod, "ExecEvalJsonExprItemCoercion"),
+ params, lengthof(params), "");
+
+ /*
+ * Will create a block for each coercion below to check
+ * whether to evaluate the coercion's expression if there's
+ * one or to skip to the end if not.
+ */
+ b_coercions = palloc((n_coercions + 1) * sizeof(LLVMBasicBlockRef));
+ for (i = 0; i < n_coercions + 1; i++)
+ b_coercions[i] =
+ l_bb_before_v(opblocks[opno + 1],
+ "op.%d.json_item_coercion.%d",
+ opno, i);
+
+ /* Jump to check first coercion */
+ LLVMBuildBr(b, b_coercions[0]);
+
+ /* Add conditional branches for individual coercion's expressions */
+ for (cstate = &jcstate->null, i = 0;
+ cstate <= &jcstate->composite;
+ cstate++, i++)
+ {
+ /* Block for this coercion */
+ LLVMPositionBuilderAtEnd(b, b_coercions[i]);
+
+ /*
+ * Jump to evaluate the coercion's expression if the
+ * address returned by ExecEvalJsonExprItemCoercion()
+ * is the same as cstate->jump_eval_expr, if it is
+ * valid (skip to end if not), else to the next
+ * coercion's block.
+ */
+ LLVMBuildCondBr(b,
+ LLVMBuildICmp(b,
+ LLVMIntEQ,
+ v_ret,
+ l_int32_const(cstate->jump_eval_expr),
+ ""),
+ cstate->jump_eval_expr >= 0 ?
+ opblocks[cstate->jump_eval_expr] :
+ opblocks[op->d.jsonexpr_item_coercion.jump_skip_item_coercion],
+ b_coercions[i + 1]);
+ }
+
+ /*
+ * A placeholder block that the last coercion's block might
+ * jump to, which unconditionally jumps to end of
+ * coercions.
+ */
+ LLVMPositionBuilderAtEnd(b, b_coercions[i]);
+ LLVMBuildBr(b, opblocks[op->d.jsonexpr_item_coercion.jump_skip_item_coercion]);
+ break;
+ }
case EEOP_LAST:
Assert(false);
break;
diff --git a/src/backend/jit/llvm/llvmjit_types.c b/src/backend/jit/llvm/llvmjit_types.c
index 373471ad27..42c0e421d3 100644
--- a/src/backend/jit/llvm/llvmjit_types.c
+++ b/src/backend/jit/llvm/llvmjit_types.c
@@ -135,6 +135,10 @@ void *referenced_functions[] =
ExecEvalXmlExpr,
ExecEvalJsonConstructor,
ExecEvalJsonIsPredicate,
+ ExecEvalJsonExprSkip,
+ ExecEvalJsonExprBehavior,
+ ExecEvalJsonExprCoercion,
+ ExecEvalJsonExprItemCoercion,
ExecEvalJson,
MakeExpandedObjectReadOnlyInternal,
slot_getmissingattrs,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..12ed0373d0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -901,7 +901,11 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
{
JsonExpr *jsexpr = (JsonExpr *) node;
- if (ExecEvalJsonNeedsSubTransaction(jsexpr, NULL))
+ /*
+ * XXX - don't really know why it makes sense to ignore the coercion
+ * part here.
+ */
+ if (ExecEvalJsonNeedsSubTransaction(jsexpr, false))
{
context->max_hazard = PROPARALLEL_UNSAFE;
return true;
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 10c7e64aab..c3fc77cf51 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -286,6 +286,8 @@ static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt,
JsonPathItem *jsp, JsonValueList *found, JsonPathBool res);
static void getJsonPathItem(JsonPathExecContext *cxt, JsonPathItem *item,
JsonbValue *value);
+static int EvalJsonPathVar(void *vars, char *varName, int varNameLen,
+ JsonbValue *val, JsonbValue *baseObject);
static void getJsonPathVariable(JsonPathExecContext *cxt,
JsonPathItem *variable, JsonbValue *value);
static int getJsonPathVariableFromJsonb(void *varsJsonb, char *varName,
@@ -2172,6 +2174,72 @@ getJsonPathItem(JsonPathExecContext *cxt, JsonPathItem *item,
}
}
+/*
+ * Evaluate a JSON path variable caching computed value.
+ */
+int
+EvalJsonPathVar(void *cxt, char *varName, int varNameLen,
+ JsonbValue *val, JsonbValue *baseObject)
+{
+ JsonPathVariableEvalContext *var = NULL;
+ List *vars = cxt;
+ ListCell *lc;
+ int id = 1;
+
+ if (!varName)
+ return list_length(vars);
+
+ foreach(lc, vars)
+ {
+ var = lfirst(lc);
+
+ if (!strncmp(var->name, varName, varNameLen))
+ break;
+
+ var = NULL;
+ id++;
+ }
+
+ if (!var)
+ return -1;
+
+ /*
+ * When belonging to a JsonExpr, path variables are computed with the
+ * JsonExpr's ExprState (var->estate is NULL), so don't need to be computed
+ * here. In some other cases, such as when the path variables belonging
+ * to a JsonTable instead, those variables must be evaluated on their own,
+ * without the enclosing JsonExpr itself needing to be evaluated, so must
+ * be handled here.
+ */
+ if (var->estate && !var->evaluated)
+ {
+ MemoryContext oldcxt = var->mcxt ?
+ MemoryContextSwitchTo(var->mcxt) : NULL;
+
+ Assert(var->econtext != NULL);
+ var->value = ExecEvalExpr(var->estate, var->econtext, &var->isnull);
+ var->evaluated = true;
+
+ if (oldcxt)
+ MemoryContextSwitchTo(oldcxt);
+ }
+ else
+ {
+ Assert(var->evaluated);
+ }
+
+ if (var->isnull)
+ {
+ val->type = jbvNull;
+ return 0;
+ }
+
+ JsonItemFromDatum(var->value, var->typid, var->typmod, val);
+
+ *baseObject = *val;
+ return id;
+}
+
/*
* Get the value of variable passed to jsonpath executor
*/
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index c8ef917ffe..64820b4cf9 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -244,7 +244,11 @@ typedef enum ExprEvalOp
EEOP_SUBPLAN,
EEOP_JSON_CONSTRUCTOR,
EEOP_IS_JSON,
- EEOP_JSONEXPR,
+ EEOP_JSONEXPR_SKIP,
+ EEOP_JSONEXPR_PATH,
+ EEOP_JSONEXPR_BEHAVIOR,
+ EEOP_JSONEXPR_COERCION,
+ EEOP_JSON_ITEM_COERCION,
/* aggregation related nodes */
EEOP_AGG_STRICT_DESERIALIZE,
@@ -692,12 +696,53 @@ typedef struct ExprEvalStep
JsonIsPredicate *pred; /* original expression node */
} is_json;
- /* for EEOP_JSONEXPR */
+ /* for EEOP_JSONEXPR_PATH */
struct
{
struct JsonExprState *jsestate;
} jsonexpr;
+ /* for EEOP_JSONEXPR_SKIP */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+
+ /* See ExecEvalJsonExprSkip() */
+ int jump_coercion;
+ int jump_passing_args;
+ } jsonexpr_skip;
+
+ /* for EEOP_JSONEXPR_BEHAVIOR */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+
+ /* See ExecEvalJsonExprBehavior() */
+ int jump_onerror_default;
+ int jump_onempty_default;
+ int jump_coercion;
+ int jump_skip_coercion;
+ } jsonexpr_behavior;
+
+ /* for EEOP_JSONEXPR_COERCION */
+ struct
+ {
+ /* Same as jsonexpr.jsestate */
+ struct JsonExprState *jsestate;
+
+ /* See ExecEvalJsonExprCoercion() */
+ int jump_coercion_error;
+ int jump_coercion_done;
+ } jsonexpr_coercion;
+
+ struct
+ {
+ /* See ExecEvalJsonExprItemCoercion() */
+ struct JsonItemCoercionsState *jcstate;
+ int jump_skip_item_coercion;
+ } jsonexpr_item_coercion;
} d;
} ExprEvalStep;
@@ -757,49 +802,117 @@ typedef struct JsonConstructorExprState
int nargs;
} JsonConstructorExprState;
+/*
+ * Information computed before evaluating a JsonExpr expression.
+ */
+typedef struct JsonExprPreEvalState
+{
+ /* value/isnull for JsonExpr.formatted_expr */
+ NullableDatum formatted_expr;
+
+ /* value/isnull for JsonExpr.pathspec */
+ NullableDatum pathspec;
+
+ /* JsonPathVariableEvalContext entries for JsonExpr.passing_values */
+ List *args;
+} JsonExprPreEvalState;
+
+/*
+ * State for some post-JsonExpr-evaluation processing steps that gets filled
+ * in JsonExpr evaluation.
+ */
+typedef struct JsonExprPostEvalState
+{
+ /* Is JSON item empty? */
+ bool empty;
+
+ /* Did JSON item evaluation cause an error? */
+ bool error;
+
+ /* Did coercion evaluation cause an error? */
+ bool coercion_error;
+
+ /* Has the result been coerced properly? */
+ bool coercion_done;
+
+ /* Use a sub-transaction when evaluating the coercion */
+ bool coercion_use_subtrans;
+
+ /* Cache for json_populate_type() called for coercion in some cases */
+ void *cache;
+
+ /*
+ * State for evaluating a JSON item coercion. Points to either
+ * JsonExprState.coercions or JsonExprState.result_coercion; might be
+ * set to NULL if it is to be left for ExecEvalJsonCoercion() to decide
+ * how to perform the coercion.
+ */
+ ExprState *coercion;
+} JsonExprPostEvalState;
+
/* EEOP_JSONEXPR state, too big to inline */
typedef struct JsonExprState
{
JsonExpr *jsexpr; /* original expression node */
+ JsonExprPreEvalState pre_eval;
+ JsonExprPostEvalState post_eval;
+
+ /*
+ * Should use a sub-transaction for path evaluation and subsequent
+ * coercion evaluation, if any?
+ */
+ bool needSubtrans;
+ bool coercionNeedSubtrans;
+
struct
{
- FmgrInfo func; /* typinput function for output type */
+ FmgrInfo *finfo; /* typinput function for output type */
Oid typioparam;
} input; /* I/O info for output type */
- NullableDatum
- *formatted_expr, /* formatted context item value */
- *res_expr, /* result item */
- *coercion_expr, /* input for JSON item coercion */
- *pathspec; /* path specification value */
+ /*
+ * Either of the following two is used by ExecEvalJsonCoercion() to apply
+ * coercion to the final result if needed.
+ */
+ ExprState *result_coercion;
+ ExprState *coercions;
+} JsonExprState;
- ExprState *result_expr; /* coerced to output type */
- ExprState *default_on_empty; /* ON EMPTY DEFAULT expression */
- ExprState *default_on_error; /* ON ERROR DEFAULT expression */
- List *args; /* passing arguments */
+/*
+ * State for a given member of JsonItemCoercions.
+ */
+typedef struct JsonItemCoercionState
+{
+ /* Expression used to evaluate the coercion */
+ JsonCoercion *coercion;
- void *cache; /* cache for json_populate_type() */
+ /* ExprEvalStep to compute this coercion's expression */
+ int jump_eval_expr;
+} JsonItemCoercionState;
- struct JsonCoercionsState
- {
- struct JsonCoercionState
- {
- JsonCoercion *coercion; /* coercion expression */
- ExprState *estate; /* coercion expression state */
- } null,
- string,
- numeric ,
- boolean,
- date,
- time,
- timetz,
- timestamp,
- timestamptz,
- composite;
- } coercions; /* states for coercion from SQL/JSON item
- * types directly to the output type */
-} JsonExprState;
+/*
+ * State for evaluating the coercion for a given JSON item using one of
+ * the following coercions.
+ *
+ * Note that while ExecInitExprRec() for JsonItemCoercions will initialize
+ * ExprEvalSteps for all of the members that need it, only one will get run
+ * during a given evaluation of the enclosing JsonExpr depending on the type
+ * of the result JSON item.
+ */
+typedef struct JsonItemCoercionsState
+{
+ JsonItemCoercionState null;
+ JsonItemCoercionState string;
+ JsonItemCoercionState numeric;
+ JsonItemCoercionState boolean;
+ JsonItemCoercionState date;
+ JsonItemCoercionState time;
+ JsonItemCoercionState timetz;
+ JsonItemCoercionState timestamp;
+ JsonItemCoercionState timestamptz;
+ JsonItemCoercionState composite;
+} JsonItemCoercionsState;
/* functions in execExpr.c */
extern void ExprEvalPushStep(ExprState *es, const ExprEvalStep *s);
@@ -860,14 +973,18 @@ extern void ExecEvalSysVar(ExprState *state, ExprEvalStep *op,
ExprContext *econtext, TupleTableSlot *slot);
extern void ExecEvalJsonConstructor(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
+extern int ExecEvalJsonExprSkip(ExprState *state, ExprEvalStep *op);
+extern int ExecEvalJsonExprBehavior(ExprState *state, ExprEvalStep *op);
+extern int ExecEvalJsonExprCoercion(ExprState *state, ExprEvalStep *op,
+ ExprContext *econtext);
+extern int ExecEvalJsonExprItemCoercion(ExprState *state, ExprEvalStep *op);
extern void ExecEvalJson(ExprState *state, ExprEvalStep *op,
ExprContext *econtext);
-extern Datum ExecPrepareJsonItemCoercion(struct JsonbValue *item,
- JsonReturning *returning,
- struct JsonCoercionsState *coercions,
- struct JsonCoercionState **pjcstate);
-extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr,
- struct JsonCoercionsState *);
+JsonCoercion *ExecGetJsonItemCoercion(struct JsonbValue *item, JsonItemCoercions *coercions);
+extern Datum ExecPrepareJsonItemCoercion(Datum itemval,
+ JsonItemCoercionsState *coercions,
+ JsonItemCoercionState **pjcstate);
+extern bool ExecEvalJsonNeedsSubTransaction(JsonExpr *jsexpr, bool consider_coerce);
extern Datum ExecEvalExprPassingCaseValue(ExprState *estate,
ExprContext *econtext, bool *isnull,
Datum caseval_datum,
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index bae466b523..6bdd9f5121 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -69,8 +69,10 @@ typedef enum
/* Convenience macros */
#define DatumGetJsonbP(d) ((Jsonb *) PG_DETOAST_DATUM(d))
+#define DatumGetJsonbValueP(d) ((JsonbValue *) DatumGetPointer(d))
#define DatumGetJsonbPCopy(d) ((Jsonb *) PG_DETOAST_DATUM_COPY(d))
#define JsonbPGetDatum(p) PointerGetDatum(p)
+#define JsonbValuePGetDatum(p) PointerGetDatum(p)
#define PG_GETARG_JSONB_P(x) DatumGetJsonbP(PG_GETARG_DATUM(x))
#define PG_GETARG_JSONB_P_COPY(x) DatumGetJsonbPCopy(PG_GETARG_DATUM(x))
#define PG_RETURN_JSONB_P(x) PG_RETURN_POINTER(x)
diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h
index 8e79b8dc9f..fa1d3eae6c 100644
--- a/src/include/utils/jsonpath.h
+++ b/src/include/utils/jsonpath.h
@@ -280,9 +280,6 @@ extern Datum JsonPathQuery(Datum jb, JsonPath *jp, JsonWrapper wrapper,
extern JsonbValue *JsonPathValue(Datum jb, JsonPath *jp, bool *empty,
bool *error, List *vars);
-extern int EvalJsonPathVar(void *vars, char *varName, int varNameLen,
- JsonbValue *val, JsonbValue *baseObject);
-
extern PGDLLIMPORT const TableFuncRoutine JsonbTableRoutine;
#endif
--
2.35.3
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-08-09 13:59 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
1 sibling, 1 reply; 75+ messages in thread
From: Jonathan S. Katz @ 2022-08-09 13:59 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Amit Langote <amitlangote09@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Andrew Dunstan <andrew@dunslane.net>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 8/5/22 4:36 PM, Andres Freund wrote:
> Hi,
>
> I tried to look into some of the questions from Amit, but I have e.g. no idea
> what exactly the use of subtransactions tries to achieve - afaics 1a36bc9dba8
> is the first patch to introduce needing to evaluate parts expressions in a
> subtransaction - but there's not a single comment explaining why.
>
> It's really hard to understand the new json code. It's a substantial amount of
> new infrastructure, without any design documentation that I can find. And it's
> not like it's just code that's equivalent to nearby stuff. To me this falls
> way below our standards and I think it's *months* of work to fix.
>
> Even just the expresion evaluation code: EvalJsonPathVar(),
> ExecEvalJsonConstructor(), ExecEvalJsonExpr(), ExecEvalJson(). There's one
> layer of subtransactions in one of the paths in ExecEvalJsonExpr(), another in
> ExecEvalJson(). Some paths of ExecEvalJsonExpr() go through subtransactions,
> others don't.
>
> It's one thing for a small set of changes to be of this quality. But this is
> pretty large - a quick summing of diffstat ends up with about 17k lines added,
> of which ~2.5k are docs, ~4.8k are tests.
The RMT met today to discuss the state of this open item surrounding the
SQL/JSON feature set. We discussed the specific concerns raised about
the code and debated four different options:
1. Do nothing and continue with the community process of stabilizing
the code without significant redesign
2. Recommend holding up the v15 release to allow for the code to be
redesigned and fixed (as based on Andres' estimates, this would push the
release out several months).
3. Revert the problematic parts of the code but try to include some
of the features in the v15 release (e.g. JSON_TABLE)
4. Revert the feature set and redesign and try to include for v16
Based on the concerns raised, the RMT is recommending option #4, to
revert the SQL/JSON changes for v15, and come back with a redesign for v16.
If folks think there are some bits we can include in v15, we can
consider option #3. (Personally, I would like to see if we can keep
JSON_TABLE, but if there is too much overlap with the problematic
portions of the code I am fine with waiting for v16).
At this stage in the release process coupled with the concerns, we're a
bit worried about introducing changes that are unpredictable in terms of
stability and maintenance. We also do not want to hold up the release
while this feature set is goes through a redesign without agreement on
what such a design would look like as well as a timeline.
Note that the above are the RMT's recommendations; while the RMT can
explicitly call for a revert, we want to first guide the discussion on
the best path forward knowing the challenges for including these
features in v15.
Thanks,
Jonathan
Attachments:
OpenPGP_signature (application/pgp-signature, 839B) download
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
@ 2022-08-09 15:03 ` Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
0 siblings, 1 reply; 75+ messages in thread
From: Andrew Dunstan @ 2022-08-09 15:03 UTC (permalink / raw)
To: Jonathan S. Katz <jkatz@postgresql.org>; Andres Freund <andres@anarazel.de>; Amit Langote <amitlangote09@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-08-09 Tu 09:59, Jonathan S. Katz wrote:
> On 8/5/22 4:36 PM, Andres Freund wrote:
>> Hi,
>>
>> I tried to look into some of the questions from Amit, but I have e.g.
>> no idea
>> what exactly the use of subtransactions tries to achieve - afaics
>> 1a36bc9dba8
>> is the first patch to introduce needing to evaluate parts expressions
>> in a
>> subtransaction - but there's not a single comment explaining why.
>>
>> It's really hard to understand the new json code. It's a substantial
>> amount of
>> new infrastructure, without any design documentation that I can find.
>> And it's
>> not like it's just code that's equivalent to nearby stuff. To me this
>> falls
>> way below our standards and I think it's *months* of work to fix.
>>
>> Even just the expresion evaluation code: EvalJsonPathVar(),
>> ExecEvalJsonConstructor(), ExecEvalJsonExpr(), ExecEvalJson().
>> There's one
>> layer of subtransactions in one of the paths in ExecEvalJsonExpr(),
>> another in
>> ExecEvalJson(). Some paths of ExecEvalJsonExpr() go through
>> subtransactions,
>> others don't.
>>
>> It's one thing for a small set of changes to be of this quality. But
>> this is
>> pretty large - a quick summing of diffstat ends up with about 17k
>> lines added,
>> of which ~2.5k are docs, ~4.8k are tests.
>
> The RMT met today to discuss the state of this open item surrounding
> the SQL/JSON feature set. We discussed the specific concerns raised
> about the code and debated four different options:
>
> 1. Do nothing and continue with the community process of stabilizing
> the code without significant redesign
>
> 2. Recommend holding up the v15 release to allow for the code to be
> redesigned and fixed (as based on Andres' estimates, this would push
> the release out several months).
>
> 3. Revert the problematic parts of the code but try to include some
> of the features in the v15 release (e.g. JSON_TABLE)
>
> 4. Revert the feature set and redesign and try to include for v16
>
> Based on the concerns raised, the RMT is recommending option #4, to
> revert the SQL/JSON changes for v15, and come back with a redesign for
> v16.
>
> If folks think there are some bits we can include in v15, we can
> consider option #3. (Personally, I would like to see if we can keep
> JSON_TABLE, but if there is too much overlap with the problematic
> portions of the code I am fine with waiting for v16).
>
> At this stage in the release process coupled with the concerns, we're
> a bit worried about introducing changes that are unpredictable in
> terms of stability and maintenance. We also do not want to hold up the
> release while this feature set is goes through a redesign without
> agreement on what such a design would look like as well as a timeline.
>
> Note that the above are the RMT's recommendations; while the RMT can
> explicitly call for a revert, we want to first guide the discussion on
> the best path forward knowing the challenges for including these
> features in v15.
>
>
I very much doubt option 3 is feasible. The parts that are controversial
go back at least in part to the first patches of the series. Trying to
salvage anything would almost certainly be more disruptive than trying
to fix it.
I'm not sure what the danger is to stability, performance or correctness
in applying the changes Amit has proposed for release 15. But if that
danger is judged to be too great then I agree we should revert.
I should add that I'm very grateful to Amit for his work, and I'm sure
it isn't wasted effort, whatever the decision.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-08-09 18:04 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 18:57 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 19:17 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 75+ messages in thread
From: Jonathan S. Katz @ 2022-08-09 18:04 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; Andres Freund <andres@anarazel.de>; Amit Langote <amitlangote09@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 8/9/22 11:03 AM, Andrew Dunstan wrote:
>
> On 2022-08-09 Tu 09:59, Jonathan S. Katz wrote:
>> The RMT met today to discuss the state of this open item surrounding
>> the SQL/JSON feature set. We discussed the specific concerns raised
>> about the code and debated four different options:
>>
>> 1. Do nothing and continue with the community process of stabilizing
>> the code without significant redesign
>>
>> 2. Recommend holding up the v15 release to allow for the code to be
>> redesigned and fixed (as based on Andres' estimates, this would push
>> the release out several months).
>>
>> 3. Revert the problematic parts of the code but try to include some
>> of the features in the v15 release (e.g. JSON_TABLE)
>>
>> 4. Revert the feature set and redesign and try to include for v16
>>
>> Based on the concerns raised, the RMT is recommending option #4, to
>> revert the SQL/JSON changes for v15, and come back with a redesign for
>> v16.
>>
>> If folks think there are some bits we can include in v15, we can
>> consider option #3. (Personally, I would like to see if we can keep
>> JSON_TABLE, but if there is too much overlap with the problematic
>> portions of the code I am fine with waiting for v16).
>>
>> At this stage in the release process coupled with the concerns, we're
>> a bit worried about introducing changes that are unpredictable in
>> terms of stability and maintenance. We also do not want to hold up the
>> release while this feature set is goes through a redesign without
>> agreement on what such a design would look like as well as a timeline.
>>
>> Note that the above are the RMT's recommendations; while the RMT can
>> explicitly call for a revert, we want to first guide the discussion on
>> the best path forward knowing the challenges for including these
>> features in v15.
>>
>>
>
> I very much doubt option 3 is feasible. The parts that are controversial
> go back at least in part to the first patches of the series. Trying to
> salvage anything would almost certainly be more disruptive than trying
> to fix it.
>
> I'm not sure what the danger is to stability, performance or correctness
> in applying the changes Amit has proposed for release 15. But if that
> danger is judged to be too great then I agree we should revert.
Speaking personally, I would like to see what we could do to include
support for this batch of the SQL/JSON features in v15. What is included
looks like it closes most of the gap on what we've been missing
syntactically since the standard was adopted, and the JSON_TABLE work is
very convenient for converting JSON data into a relational format. I
believe having this feature set is important for maintaining standards
compliance, interoperability, tooling support, and general usability.
Plus, JSON still seems to be pretty popular :)
Rereading the thread for the umpteenth time, I have seen Amit working
through Andres' concerns. From my read, the ones that seem pressing are:
* Lack of design documentation, which may be leading to some of the
general design concerns
* The use of substransactions within the executor, though docs
explaining the decisions on that could alleviate it (I realize this is a
big topic and any summary I give won't capture the full nuance)
* Debate on how to handle the type coercions
(Please correct me if I've missed anything).
I hope that these can be addressed satisfactorily in a reasonable (read:
now a much shorter) timeframe so we can include the SQL/JSON work in v15.
With my RMT hat on, the issue is that we're now at beta 3 and we still
have not not reached a resolution on this open item. Even if it were
committed tomorrow, we would definitely need a beta 4, and we would want
to let the code bake a bit more to ensure we get adequate test coverage
on it. This would likely put the release date into October, presuming we
have not found any other issues that could cause a release delay.
With my advocacy hat on, we're at the point in the release cycle where I
kick off the GA release process (e.g. announcement drafting). Not
knowing the final status of a feature that's likely to be highlighted
makes it difficult to write said release as well as kick off the other
machinery (e.g. translations). If there is at least a decision on next
steps, I can adjust the GA release process timeline.
> I should add that I'm very grateful to Amit for his work, and I'm sure
> it isn't wasted effort, whatever the decision.
+1. While I've been quiet on this thread to date, I have definitely seen
Amit working hard on addressing the concerns.
Thanks,
Jonathan
Attachments:
OpenPGP_signature (application/pgp-signature, 839B) download
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
@ 2022-08-09 18:57 ` Andres Freund <andres@anarazel.de>
2022-08-09 19:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-10 13:27 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
1 sibling, 2 replies; 75+ messages in thread
From: Andres Freund @ 2022-08-09 18:57 UTC (permalink / raw)
To: Jonathan S. Katz <jkatz@postgresql.org>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-08-09 14:04:48 -0400, Jonathan S. Katz wrote:
> On 8/9/22 11:03 AM, Andrew Dunstan wrote:
> >
> > On 2022-08-09 Tu 09:59, Jonathan S. Katz wrote:
>
> > > The RMT met today to discuss the state of this open item surrounding
> > > the SQL/JSON feature set. We discussed the specific concerns raised
> > > about the code and debated four different options:
> > >
> > > 1. Do nothing and continue with the community process of stabilizing
> > > the code without significant redesign
> > >
> > > 2. Recommend holding up the v15 release to allow for the code to be
> > > redesigned and fixed (as based on Andres' estimates, this would push
> > > the release out several months).
Obviously that's a question of the resources brought to bear.
From my angle: I've obviously some of my own work to take care of as well, but
it's also just hard to improve something that includes a lot of undocumented
design decisions.
> > > 3. Revert the problematic parts of the code but try to include some
> > > of the features in the v15 release (e.g. JSON_TABLE)
> > I very much doubt option 3 is feasible. The parts that are controversial
> > go back at least in part to the first patches of the series. Trying to
> > salvage anything would almost certainly be more disruptive than trying
> > to fix it.
Agreed.
> > > 4. Revert the feature set and redesign and try to include for v16
> > >
> > > Based on the concerns raised, the RMT is recommending option #4, to
> > > revert the SQL/JSON changes for v15, and come back with a redesign for
> > > v16.
> > >
> > > If folks think there are some bits we can include in v15, we can
> > > consider option #3. (Personally, I would like to see if we can keep
> > > JSON_TABLE, but if there is too much overlap with the problematic
> > > portions of the code I am fine with waiting for v16).
> > >
> > > At this stage in the release process coupled with the concerns, we're
> > > a bit worried about introducing changes that are unpredictable in
> > > terms of stability and maintenance. We also do not want to hold up the
> > > release while this feature set is goes through a redesign without
> > > agreement on what such a design would look like as well as a timeline.
> > >
> > > Note that the above are the RMT's recommendations; while the RMT can
> > > explicitly call for a revert, we want to first guide the discussion on
> > > the best path forward knowing the challenges for including these
> > > features in v15.
Unless we decide on 4 immediately, I think it might be worth starting a
separate thread to get more attention. The subject doesn't necessarily have
everyone follow along.
> > I'm not sure what the danger is to stability, performance or correctness
> > in applying the changes Amit has proposed for release 15. But if that
> > danger is judged to be too great then I agree we should revert.
My primary problem is that as-is the code is nearly unmaintainable. It's hard
for Amit to fix that, given that he's not one of the original authors.
> Rereading the thread for the umpteenth time, I have seen Amit working
> through Andres' concerns. From my read, the ones that seem pressing are:
>
> * Lack of design documentation, which may be leading to some of the general
> design concerns
> * The use of substransactions within the executor, though docs explaining
> the decisions on that could alleviate it (I realize this is a big topic and
> any summary I give won't capture the full nuance)
I don't think subtransactions per-se are a fundamental problem. I think the
error handling implementation is ridiculously complicated, and while I started
to hack on improving it, I stopped when I really couldn't understand what
errors it actually needs to handle when and why.
> * Debate on how to handle the type coercions
That's partially related to the error handling issue above.
One way this code could be drastically simplified is to force all
type-coercions to go through the "io coercion" path, which could be
implemented as a single execution step (which thus could trivially
start/finish a subtransaction) and would remove a lot of the complicated code
around coercions.
> > I should add that I'm very grateful to Amit for his work, and I'm sure
> > it isn't wasted effort, whatever the decision.
>
> +1. While I've been quiet on this thread to date, I have definitely seen
> Amit working hard on addressing the concerns.
+1
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 18:57 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-08-09 19:59 ` Jonathan S. Katz <jkatz@postgresql.org>
1 sibling, 0 replies; 75+ messages in thread
From: Jonathan S. Katz @ 2022-08-09 19:59 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 8/9/22 2:57 PM, Andres Freund wrote:
> Hi,
>
> On 2022-08-09 14:04:48 -0400, Jonathan S. Katz wrote:
>>>> 2. Recommend holding up the v15 release to allow for the code to be
>>>> redesigned and fixed (as based on Andres' estimates, this would push
>>>> the release out several months).
>
> Obviously that's a question of the resources brought to bear.
>
> From my angle: I've obviously some of my own work to take care of as well, but
> it's also just hard to improve something that includes a lot of undocumented
> design decisions.
*nod*
>>>> 4. Revert the feature set and redesign and try to include for v16
> Unless we decide on 4 immediately, I think it might be worth starting a
> separate thread to get more attention. The subject doesn't necessarily have
> everyone follow along.
*nod* I'll do this shortly.
>> Rereading the thread for the umpteenth time, I have seen Amit working
>> through Andres' concerns. From my read, the ones that seem pressing are:
>>
>> * Lack of design documentation, which may be leading to some of the general
>> design concerns
>
>> * The use of substransactions within the executor, though docs explaining
>> the decisions on that could alleviate it (I realize this is a big topic and
>> any summary I give won't capture the full nuance)
>
> I don't think subtransactions per-se are a fundamental problem. I think the
> error handling implementation is ridiculously complicated, and while I started
> to hack on improving it, I stopped when I really couldn't understand what
> errors it actually needs to handle when and why.
Ah, thanks for the clarification. That makes sense.
>> * Debate on how to handle the type coercions
>
> That's partially related to the error handling issue above.
>
> One way this code could be drastically simplified is to force all
> type-coercions to go through the "io coercion" path, which could be
> implemented as a single execution step (which thus could trivially
> start/finish a subtransaction) and would remove a lot of the complicated code
> around coercions.
If we went down this path, would this make you feel more comfortable
with including this work in the v15 release?
Thanks,
Jonathan
Attachments:
OpenPGP_signature (application/pgp-signature, 839B) download
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 18:57 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-08-10 13:27 ` Amit Langote <amitlangote09@gmail.com>
2022-08-11 17:08 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
1 sibling, 1 reply; 75+ messages in thread
From: Amit Langote @ 2022-08-10 13:27 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Jonathan S. Katz <jkatz@postgresql.org>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On Wed, Aug 10, 2022 at 3:57 AM Andres Freund <andres@anarazel.de> wrote:
> One way this code could be drastically simplified is to force all
> type-coercions to go through the "io coercion" path, which could be
> implemented as a single execution step (which thus could trivially
> start/finish a subtransaction) and would remove a lot of the complicated code
> around coercions.
Could you please clarify how you think we might do the io coercion
wrapped with a subtransaction all as a single execution step? I
would've thought that we couldn't do the sub-transaction without
leaving ExecInterpExpr() anyway, so maybe you meant the io coercion
itself was done using some code outside ExecInterpExpr()?
The current JsonExpr code does it by recursively calling
ExecInterpExpr() using the nested ExprState expressly for the
coercion.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 18:57 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-10 13:27 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
@ 2022-08-11 17:08 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-11 17:17 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 75+ messages in thread
From: Jonathan S. Katz @ 2022-08-11 17:08 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; Andres Freund <andres@anarazel.de>; +Cc: Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 8/10/22 9:27 AM, Amit Langote wrote:
> On Wed, Aug 10, 2022 at 3:57 AM Andres Freund <andres@anarazel.de> wrote:
>> One way this code could be drastically simplified is to force all
>> type-coercions to go through the "io coercion" path, which could be
>> implemented as a single execution step (which thus could trivially
>> start/finish a subtransaction) and would remove a lot of the complicated code
>> around coercions.
>
> Could you please clarify how you think we might do the io coercion
> wrapped with a subtransaction all as a single execution step? I
> would've thought that we couldn't do the sub-transaction without
> leaving ExecInterpExpr() anyway, so maybe you meant the io coercion
> itself was done using some code outside ExecInterpExpr()?
>
> The current JsonExpr code does it by recursively calling
> ExecInterpExpr() using the nested ExprState expressly for the
> coercion.
With RMT hat on, Andres do you have any thoughts on this?
Thanks,
Jonathan
Attachments:
OpenPGP_signature (application/pgp-signature, 839B) download
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 18:57 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-10 13:27 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-11 17:08 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
@ 2022-08-11 17:17 ` Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 75+ messages in thread
From: Andres Freund @ 2022-08-11 17:17 UTC (permalink / raw)
To: Jonathan S. Katz <jkatz@postgresql.org>; +Cc: Amit Langote <amitlangote09@gmail.com>; Andrew Dunstan <andrew@dunslane.net>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-08-11 13:08:27 -0400, Jonathan S. Katz wrote:
> On 8/10/22 9:27 AM, Amit Langote wrote:
> > On Wed, Aug 10, 2022 at 3:57 AM Andres Freund <andres@anarazel.de> wrote:
> > > One way this code could be drastically simplified is to force all
> > > type-coercions to go through the "io coercion" path, which could be
> > > implemented as a single execution step (which thus could trivially
> > > start/finish a subtransaction) and would remove a lot of the complicated code
> > > around coercions.
> >
> > Could you please clarify how you think we might do the io coercion
> > wrapped with a subtransaction all as a single execution step? I
> > would've thought that we couldn't do the sub-transaction without
> > leaving ExecInterpExpr() anyway, so maybe you meant the io coercion
> > itself was done using some code outside ExecInterpExpr()?
> >
> > The current JsonExpr code does it by recursively calling
> > ExecInterpExpr() using the nested ExprState expressly for the
> > coercion.
The basic idea is to rip out all the type-dependent stuff out and replace it
with a single JSON_IOCERCE step, which has a parameter about whether to wrap
things in a subtransaction or not. That step would always perform the coercion
by calling the text output function of the input and the text input function
of the output.
> With RMT hat on, Andres do you have any thoughts on this?
I think I need to prototype how it'd look like to give a more detailed
answer. I have a bunch of meetings over the next few hours, but after that I
can give it a shot.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
@ 2022-08-09 19:17 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-08-09 19:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 75+ messages in thread
From: Tom Lane @ 2022-08-09 19:17 UTC (permalink / raw)
To: Jonathan S. Katz <jkatz@postgresql.org>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Andres Freund <andres@anarazel.de>; Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> Speaking personally, I would like to see what we could do to include
> support for this batch of the SQL/JSON features in v15. What is included
> looks like it closes most of the gap on what we've been missing
> syntactically since the standard was adopted, and the JSON_TABLE work is
> very convenient for converting JSON data into a relational format. I
> believe having this feature set is important for maintaining standards
> compliance, interoperability, tooling support, and general usability.
> Plus, JSON still seems to be pretty popular :)
> ...
> I hope that these can be addressed satisfactorily in a reasonable (read:
> now a much shorter) timeframe so we can include the SQL/JSON work in v15.
We have delayed releases for $COOL_FEATURE in the past, and I think
our batting average on that is still .000: not once has it worked out
well. I think we're better off getting the pain over with quickly,
so I regretfully vote for revert. And for a full redesign/rewrite
before we try again; based on Andres' comments, it needs that.
regards, tom lane
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 19:17 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-08-09 19:22 ` Andres Freund <andres@anarazel.de>
2022-08-09 19:50 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
0 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-08-09 19:22 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Jonathan S. Katz <jkatz@postgresql.org>; Andrew Dunstan <andrew@dunslane.net>; Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-08-09 15:17:44 -0400, Tom Lane wrote:
> We have delayed releases for $COOL_FEATURE in the past, and I think
> our batting average on that is still .000: not once has it worked out
> well.
I think it semi worked when jsonb (?) first went in - it took a while and a
lot of effort from a lot of people, but in the end we made it work, and it was
a success from our user's perspectives, I think. OTOH, it's not a great sign
this is around json again...
- Andres
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 19:17 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-08-09 19:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-08-09 19:50 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 20:15 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
0 siblings, 1 reply; 75+ messages in thread
From: Jonathan S. Katz @ 2022-08-09 19:50 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 8/9/22 3:22 PM, Andres Freund wrote:
> Hi,
>
> On 2022-08-09 15:17:44 -0400, Tom Lane wrote:
>> We have delayed releases for $COOL_FEATURE in the past, and I think
>> our batting average on that is still .000: not once has it worked out
>> well.
>
> I think it semi worked when jsonb (?) first went in - it took a while and a
> lot of effort from a lot of people, but in the end we made it work, and it was
> a success from our user's perspectives, I think.
Yeah, this was the example I was thinking of. To continue with the
baseball analogy, it was a home-run from a PR perspective, and I can say
as a power user at the time, the 9.4 JSONB representation worked well
for my use case. Certainly newer functionality has made JSON easier to
work with in PG.
(I can't remember what the 9.5 hold up was).
The cases where we either delayed/punted on $COOL_FEATURE that cause me
concern are the ones where we say "OK, well fix this in the next
release" and we are then waiting, 2, 3, 4 releases for the work to be
completed. And to be clear, I'm thinking of this as "known issues" vs.
"iterating towards the whole solution".
> OTOH, it's not a great sign this is around json again...
Yeah, I was thinking about that too.
Per Andres comment upthread, let's open a new thread to discuss the
SQL/JSON + v15 topic to improve visibility and get more feedback. I can
do that shortly.
We can continue with the technical discussion in here.
Jonathan
Attachments:
OpenPGP_signature (application/pgp-signature, 839B) download
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 19:17 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-08-09 19:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 19:50 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
@ 2022-08-09 20:15 ` Andrew Dunstan <andrew@dunslane.net>
2022-08-09 20:19 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
0 siblings, 1 reply; 75+ messages in thread
From: Andrew Dunstan @ 2022-08-09 20:15 UTC (permalink / raw)
To: Jonathan S. Katz <jkatz@postgresql.org>; Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-08-09 Tu 15:50, Jonathan S. Katz wrote:
> On 8/9/22 3:22 PM, Andres Freund wrote:
>> Hi,
>>
>> On 2022-08-09 15:17:44 -0400, Tom Lane wrote:
>>> We have delayed releases for $COOL_FEATURE in the past, and I think
>>> our batting average on that is still .000: not once has it worked out
>>> well.
>>
>> I think it semi worked when jsonb (?) first went in - it took a while
>> and a
>> lot of effort from a lot of people, but in the end we made it work,
>> and it was
>> a success from our user's perspectives, I think.
>
> Yeah, this was the example I was thinking of. To continue with the
> baseball analogy, it was a home-run from a PR perspective, and I can
> say as a power user at the time, the 9.4 JSONB representation worked
> well for my use case. Certainly newer functionality has made JSON
> easier to work with in PG.
>
> (I can't remember what the 9.5 hold up was).
>
> The cases where we either delayed/punted on $COOL_FEATURE that cause
> me concern are the ones where we say "OK, well fix this in the next
> release" and we are then waiting, 2, 3, 4 releases for the work to be
> completed. And to be clear, I'm thinking of this as "known issues" vs.
> "iterating towards the whole solution".
Where we ended up with jsonb was a long way from where we started, but
technical difficulties were largely confined because it didn't involve
anything like the parser or the expression evaluation code. Here the SQL
Standards Committee has imposed a pretty substantial technical burden on
us and the code that Andres complains of is attempting to deal with that.
>
>> OTOH, it's not a great sign this is around json again...
>
> Yeah, I was thinking about that too.
Ouch :-(
I think after 10 years of being involved with our JSON features, I'm
going to take a substantial break on that front.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 19:17 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-08-09 19:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 19:50 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 20:15 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
@ 2022-08-09 20:19 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 20:27 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
0 siblings, 1 reply; 75+ messages in thread
From: Jonathan S. Katz @ 2022-08-09 20:19 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 8/9/22 4:15 PM, Andrew Dunstan wrote:
>
> On 2022-08-09 Tu 15:50, Jonathan S. Katz wrote:
>> On 8/9/22 3:22 PM, Andres Freund wrote:
>>
>>> OTOH, it's not a great sign this is around json again...
>>
>> Yeah, I was thinking about that too.
>
>
> Ouch :-(
>
> I think after 10 years of being involved with our JSON features, I'm
> going to take a substantial break on that front.
I hope that wasn't taken as a sleight, but just an observation. There
are other feature areas where I can make similar observations. All this
work around a database system is challenging as there are many
considerations that need to be made.
You've done an awesome job driving the JSON work forward and it is
greatly appreciated.
Thanks,
Jonathan
Attachments:
OpenPGP_signature (application/pgp-signature, 839B) download
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-15 21:07 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 00:39 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 13:59 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 19:17 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-08-09 19:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-08-09 19:50 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 20:15 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-08-09 20:19 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Jonathan S. Katz <jkatz@postgresql.org>
@ 2022-08-09 20:27 ` Andrew Dunstan <andrew@dunslane.net>
0 siblings, 0 replies; 75+ messages in thread
From: Andrew Dunstan @ 2022-08-09 20:27 UTC (permalink / raw)
To: Jonathan S. Katz <jkatz@postgresql.org>; Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Amit Langote <amitlangote09@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael@paquier.xyz>; John Naylor <john.naylor@enterprisedb.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
On 2022-08-09 Tu 16:19, Jonathan S. Katz wrote:
> On 8/9/22 4:15 PM, Andrew Dunstan wrote:
>>
>> On 2022-08-09 Tu 15:50, Jonathan S. Katz wrote:
>>> On 8/9/22 3:22 PM, Andres Freund wrote:
>
>>>
>>>> OTOH, it's not a great sign this is around json again...
>>>
>>> Yeah, I was thinking about that too.
>>
>>
>> Ouch :-(
>>
>> I think after 10 years of being involved with our JSON features, I'm
>> going to take a substantial break on that front.
>
> I hope that wasn't taken as a sleight, but just an observation. There
> are other feature areas where I can make similar observations. All
> this work around a database system is challenging as there are many
> considerations that need to be made.
>
> You've done an awesome job driving the JSON work forward and it is
> greatly appreciated.
>
>
Thanks, I appreciate that (and I wasn't fishing for compliments). It's
more that I feel a bit tired of it ... some of my colleagues will
confirm that I've been saying this for a while, so it's not spurred by
this setback.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-07-05 19:08 ` Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 75+ messages in thread
From: Andres Freund @ 2022-07-05 19:08 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Tom Lane <tgl@sss.pgh.pa.us>; David Rowley <dgrowleyml@gmail.com>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>
Hi,
On 2022-06-23 18:51:45 -0700, Andres Freund wrote:
> > Waiting for beta3 would a better move at this stage. Is somebody confident
> > enough in the patches proposed?
>
> 0001 is the one that needs to most careful analysis, I think. 0002 I'd be fine
> with pushing after reviewing it again. For 0003 David's approach might be
> better or worse, it doesn't matter much I think. 0004 is ok I think, perhaps
> with the exception of quibbling over some naming decisions?
I don't quite feel comfortable with 0001, without review by others. So my
current plan is to drop it and use get_timeout_active() "manually". We can
improve this in HEAD to remove the redundancy.
I've pushed what was 0004, will push what was 0002 with the above change in a
short while unless somebody protests PDQ. Then will look at David's edition of
my 0003.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-28 23:40 ` David Rowley <dgrowleyml@gmail.com>
2022-07-06 00:32 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 75+ messages in thread
From: David Rowley @ 2022-06-28 23:40 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
On Sat, 18 Jun 2022 at 08:06, Andres Freund <andres@anarazel.de> wrote:
> I also attached my heavily-WIP patches for the ExprEvalStep issues, I
> accidentally had only included a small part of the contents of the json fix.
I've now looked at the 0003 patch. I like the idea you have about
moving some of the additional fields into ScalarArrayOpExprHashTable.
I think the patch can even go a little further and move the hash_finfo
into there too. This means we don't need to dereference the "op" in
saop_element_hash().
To make this work, I did need to tag the ScalarArrayOpExpr into the
ExprEvalStep. That's required now since some of the initialization of
the hash function fields is delayed until
ExecEvalHashedScalarArrayOp(). We need to know the
ScalarArrayOpExpr's hashfuncid and inputcollid.
Your v2 patch did shift off some of this initialization work to
ExecEvalHashedScalarArrayOp(). The attached v3 takes that a bit
further. This saves a bit more work for ScalarArrayOpExprs that are
evaluated 0 times.
Another small thing which I considered doing was to put the
hash_fcinfo_data field as the final field in
ScalarArrayOpExprHashTable so that we could allocate the memory for
the hash_fcinfo_data in the same allocation as the
ScalarArrayOpExprHashTable. This would reduce the pointer
dereferencing done in saop_element_hash() a bit further. I just
didn't notice anywhere else where we do that for FunctionCallInfo, so
I resisted doing this.
(There was also a small bug in your patch where you mistakenly cast to
an OpExpr instead of ScalarArrayOpExpr when you were fetching the
inputcollid)
David
Attachments:
v3-0001-Remove-size-increase-in-ExprEvalStep-caused-by-ha.patch (text/plain, 7.0K) download
From 7a1e71a6b622eff72537ae86187b312d01a5e11d Mon Sep 17 00:00:00 2001
From: David Rowley <drowley@postgresql.org>
Date: Wed, 29 Jun 2022 10:43:41 +1200
Subject: [PATCH v3] Remove size increase in ExprEvalStep caused by hashed
saops
50e17ad28 increased the size of ExprEvalStep from 64 bytes up to 88 bytes.
Lots of effort was spent during the development of the current expression
evaluation code to make an instance of this struct fit on a single cache
line.
Here we remove the fn_addr field. The values from this field can be
obtained via fcinfo, just with some extra pointer dereferencing. The
extra indirection does not seem to cause any noticeable slowdowns.
Various other fields have been been moved into the
ScalarArrayOpExprHashTable struct. These fields are only used when the
ScalarArrayOpExprHashTable pointer has already been dereferenced, so no
additional pointer dereferences occur for these.
Author: Andres Freund
Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de
---
src/backend/executor/execExpr.c | 19 +------------------
src/backend/executor/execExprInterp.c | 25 +++++++++++++++++++++----
src/include/executor/execExpr.h | 7 +------
3 files changed, 23 insertions(+), 28 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 2831e7978b..2e75f3204d 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -1203,8 +1203,6 @@ ExecInitExprRec(Expr *node, ExprState *state,
FmgrInfo *finfo;
FunctionCallInfo fcinfo;
AclResult aclresult;
- FmgrInfo *hash_finfo;
- FunctionCallInfo hash_fcinfo;
Oid cmpfuncid;
/*
@@ -1262,18 +1260,6 @@ ExecInitExprRec(Expr *node, ExprState *state,
*/
if (OidIsValid(opexpr->hashfuncid))
{
- hash_finfo = palloc0(sizeof(FmgrInfo));
- hash_fcinfo = palloc0(SizeForFunctionCallInfo(1));
- fmgr_info(opexpr->hashfuncid, hash_finfo);
- fmgr_info_set_expr((Node *) node, hash_finfo);
- InitFunctionCallInfoData(*hash_fcinfo, hash_finfo,
- 1, opexpr->inputcollid, NULL,
- NULL);
-
- scratch.d.hashedscalararrayop.hash_finfo = hash_finfo;
- scratch.d.hashedscalararrayop.hash_fcinfo_data = hash_fcinfo;
- scratch.d.hashedscalararrayop.hash_fn_addr = hash_finfo->fn_addr;
-
/* Evaluate scalar directly into left function argument */
ExecInitExprRec(scalararg, state,
&fcinfo->args[0].value, &fcinfo->args[0].isnull);
@@ -1292,11 +1278,8 @@ ExecInitExprRec(Expr *node, ExprState *state,
scratch.d.hashedscalararrayop.inclause = opexpr->useOr;
scratch.d.hashedscalararrayop.finfo = finfo;
scratch.d.hashedscalararrayop.fcinfo_data = fcinfo;
- scratch.d.hashedscalararrayop.fn_addr = finfo->fn_addr;
+ scratch.d.hashedscalararrayop.saop = opexpr;
- scratch.d.hashedscalararrayop.hash_finfo = hash_finfo;
- scratch.d.hashedscalararrayop.hash_fcinfo_data = hash_fcinfo;
- scratch.d.hashedscalararrayop.hash_fn_addr = hash_finfo->fn_addr;
ExprEvalPushStep(state, &scratch);
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index eaec697bb3..399bdfd861 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -217,6 +217,8 @@ typedef struct ScalarArrayOpExprHashTable
{
saophash_hash *hashtab; /* underlying hash table */
struct ExprEvalStep *op;
+ FunctionCallInfo hash_fcinfo_data; /* arguments etc */
+ FmgrInfo *hash_finfo; /* function's lookup data */
} ScalarArrayOpExprHashTable;
/* Define parameters for ScalarArrayOpExpr hash table code generation. */
@@ -3474,13 +3476,13 @@ static uint32
saop_element_hash(struct saophash_hash *tb, Datum key)
{
ScalarArrayOpExprHashTable *elements_tab = (ScalarArrayOpExprHashTable *) tb->private_data;
- FunctionCallInfo fcinfo = elements_tab->op->d.hashedscalararrayop.hash_fcinfo_data;
+ FunctionCallInfo fcinfo = elements_tab->hash_fcinfo_data;
Datum hash;
fcinfo->args[0].value = key;
fcinfo->args[0].isnull = false;
- hash = elements_tab->op->d.hashedscalararrayop.hash_fn_addr(fcinfo);
+ hash = elements_tab->hash_finfo->fn_addr(fcinfo);
return DatumGetUInt32(hash);
}
@@ -3502,7 +3504,7 @@ saop_hash_element_match(struct saophash_hash *tb, Datum key1, Datum key2)
fcinfo->args[1].value = key2;
fcinfo->args[1].isnull = false;
- result = elements_tab->op->d.hashedscalararrayop.fn_addr(fcinfo);
+ result = elements_tab->op->d.hashedscalararrayop.finfo->fn_addr(fcinfo);
return DatumGetBool(result);
}
@@ -3549,6 +3551,7 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
/* Build the hash table on first evaluation */
if (elements_tab == NULL)
{
+ ScalarArrayOpExpr *saop;
int16 typlen;
bool typbyval;
char typalign;
@@ -3560,6 +3563,8 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
MemoryContext oldcontext;
ArrayType *arr;
+ saop = op->d.hashedscalararrayop.saop;
+
arr = DatumGetArrayTypeP(*op->resvalue);
nitems = ArrayGetNItems(ARR_NDIM(arr), ARR_DIMS(arr));
@@ -3575,6 +3580,18 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
op->d.hashedscalararrayop.elements_tab = elements_tab;
elements_tab->op = op;
+ elements_tab->hash_finfo = palloc0(sizeof(FmgrInfo));
+ fmgr_info(saop->hashfuncid, elements_tab->hash_finfo);
+ fmgr_info_set_expr((Node *) saop, elements_tab->hash_finfo);
+
+ elements_tab->hash_fcinfo_data = palloc0(SizeForFunctionCallInfo(1));
+ InitFunctionCallInfoData(*elements_tab->hash_fcinfo_data,
+ elements_tab->hash_finfo,
+ 1,
+ saop->inputcollid,
+ NULL,
+ NULL);
+
/*
* Create the hash table sizing it according to the number of elements
* in the array. This does assume that the array has no duplicates.
@@ -3669,7 +3686,7 @@ ExecEvalHashedScalarArrayOp(ExprState *state, ExprEvalStep *op, ExprContext *eco
fcinfo->args[1].value = (Datum) 0;
fcinfo->args[1].isnull = true;
- result = op->d.hashedscalararrayop.fn_addr(fcinfo);
+ result = op->d.hashedscalararrayop.finfo->fn_addr(fcinfo);
resultnull = fcinfo->isnull;
/*
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index e34db8c93c..e5e4d7fc1f 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -582,12 +582,7 @@ typedef struct ExprEvalStep
struct ScalarArrayOpExprHashTable *elements_tab;
FmgrInfo *finfo; /* function's lookup data */
FunctionCallInfo fcinfo_data; /* arguments etc */
- /* faster to access without additional indirection: */
- PGFunction fn_addr; /* actual call address */
- FmgrInfo *hash_finfo; /* function's lookup data */
- FunctionCallInfo hash_fcinfo_data; /* arguments etc */
- /* faster to access without additional indirection: */
- PGFunction hash_fn_addr; /* actual call address */
+ ScalarArrayOpExpr *saop;
} hashedscalararrayop;
/* for EEOP_XMLEXPR */
--
2.34.1
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-28 23:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
@ 2022-07-06 00:32 ` Andres Freund <andres@anarazel.de>
2022-07-06 07:52 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-07-06 00:32 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Hi,
On 2022-06-29 11:40:45 +1200, David Rowley wrote:
> On Sat, 18 Jun 2022 at 08:06, Andres Freund <andres@anarazel.de> wrote:
> > I also attached my heavily-WIP patches for the ExprEvalStep issues, I
> > accidentally had only included a small part of the contents of the json fix.
>
> I've now looked at the 0003 patch. I like the idea you have about
> moving some of the additional fields into ScalarArrayOpExprHashTable.
> I think the patch can even go a little further and move the hash_finfo
> into there too. This means we don't need to dereference the "op" in
> saop_element_hash().
Makes sense.
> To make this work, I did need to tag the ScalarArrayOpExpr into the
> ExprEvalStep. That's required now since some of the initialization of
> the hash function fields is delayed until
> ExecEvalHashedScalarArrayOp(). We need to know the
> ScalarArrayOpExpr's hashfuncid and inputcollid.
Makes sense.
> Another small thing which I considered doing was to put the
> hash_fcinfo_data field as the final field in
> ScalarArrayOpExprHashTable so that we could allocate the memory for
> the hash_fcinfo_data in the same allocation as the
> ScalarArrayOpExprHashTable. This would reduce the pointer
> dereferencing done in saop_element_hash() a bit further. I just
> didn't notice anywhere else where we do that for FunctionCallInfo, so
> I resisted doing this.
I think that'd make sense - it does add a bit of size calculation magic, but
it shouldn't be a problem. I'm fairly sure we do this in other parts of the
code.
> (There was also a small bug in your patch where you mistakenly cast to
> an OpExpr instead of ScalarArrayOpExpr when you were fetching the
> inputcollid)
Ooops.
Are you good pushing this? I'm fine with you doing so wether you adapt it
further or not.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 14:33 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-28 23:40 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-07-06 00:32 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-07-06 07:52 ` David Rowley <dgrowleyml@gmail.com>
0 siblings, 0 replies; 75+ messages in thread
From: David Rowley @ 2022-07-06 07:52 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Peter Geoghegan <pg@bowt.ie>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Thanks for looking at this.
On Wed, 6 Jul 2022 at 12:32, Andres Freund <andres@anarazel.de> wrote:
>
> On 2022-06-29 11:40:45 +1200, David Rowley wrote:
> > Another small thing which I considered doing was to put the
> > hash_fcinfo_data field as the final field in
> > ScalarArrayOpExprHashTable so that we could allocate the memory for
> > the hash_fcinfo_data in the same allocation as the
> > ScalarArrayOpExprHashTable. This would reduce the pointer
> > dereferencing done in saop_element_hash() a bit further. I just
> > didn't notice anywhere else where we do that for FunctionCallInfo, so
> > I resisted doing this.
>
> I think that'd make sense - it does add a bit of size calculation magic, but
> it shouldn't be a problem. I'm fairly sure we do this in other parts of the
> code.
I've now adjusted that. I also changed the hash_finfo field to make
it so the FmgrInfo is inline rather than a pointer. This saves an
additional dereference in saop_element_hash() and also saves a
palloc().
I had to adjust the palloc for the ScalarArrayOpExprHashTable struct
into a palloc0 due to the FmgrInfo being inlined. I considered just
zeroing out the hash_finfo portion but thought it wasn't worth the
extra code.
> Are you good pushing this? I'm fine with you doing so wether you adapt it
> further or not.
Pushed.
David
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
@ 2022-06-17 12:33 ` James Coleman <jtc331@gmail.com>
2 siblings, 0 replies; 75+ messages in thread
From: James Coleman @ 2022-06-17 12:33 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
On Thu, Jun 16, 2022 at 10:15 PM David Rowley <dgrowleyml@gmail.com> wrote:
>
> On Fri, 17 Jun 2022 at 11:31, Andres Freund <andres@anarazel.de> wrote:
> > hashedscalararrayop/EEOP_HASHED_SCALARARRAYOP is 64 bytes, even though the
> > limit is 40 bytes.
>
> > commit 50e17ad281b8d1c1b410c9833955bc80fbad4078
> > Author: David Rowley <drowley@postgresql.org>
> > Date: 2021-04-08 23:51:22 +1200
> >
> > Speedup ScalarArrayOpExpr evaluation
>
> I've put together the attached patch which removes 4 fields from the
> hashedscalararrayop portion of the struct which, once the JSON part is
> fixed, will put sizeof(ExprEvalStep) back down to 64 bytes again.
>
> The attached patch causes some extra pointer dereferencing to perform
> a hashed saop step, so I tested the performance on f4fb45d15 (prior to
> the JSON patch that pushed the sizeof(ExprEvalStep) up further. I
> found:
>
> setup:
> create table a (a int);
> insert into a select x from generate_series(1000000,2000000) x;
>
> bench.sql
> select * from a where a in(1,2,3,4,5,6,7,8,9,10);
>
> f4fb45d15 + reduce_sizeof_hashedsaop_ExprEvalStep.patch
> drowley@amd3990x:~$ pgbench -n -f bench.sql -T 60 -M prepared postgres
> tps = 44.841851 (without initial connection time)
> tps = 44.986472 (without initial connection time)
> tps = 44.944315 (without initial connection time)
>
> f4fb45d15
> drowley@amd3990x:~$ pgbench -n -f bench.sql -T 60 -M prepared postgres
> tps = 44.446127 (without initial connection time)
> tps = 44.614134 (without initial connection time)
> tps = 44.895011 (without initial connection time)
>
> (Patched is ~0.61% faster here)
>
> So, there appears to be no performance regression due to the extra
> indirection. There's maybe even some gains due to the smaller step
> size.
I didn't see that comment when working on this (it's quite a long
unioned struct; I concur on adding an assert to catch it).
This patch looks very reasonable to me though.
James Coleman
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
@ 2022-06-17 17:21 ` Andres Freund <andres@anarazel.de>
2022-06-28 19:18 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2 siblings, 1 reply; 75+ messages in thread
From: Andres Freund @ 2022-06-17 17:21 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
Hi,
On 2022-06-17 14:14:54 +1200, David Rowley wrote:
> I've put together the attached patch which removes 4 fields from the
> hashedscalararrayop portion of the struct which, once the JSON part is
> fixed, will put sizeof(ExprEvalStep) back down to 64 bytes again.
> The attached patch causes some extra pointer dereferencing to perform
> a hashed saop step, so I tested the performance on f4fb45d15 (prior to
> the JSON patch that pushed the sizeof(ExprEvalStep) up further. I
> found:
What do you think about the approach prototyped in my patch to move the hash
FunctionCallInfo into the element_tab? With a tiny bit more work that should
reduce the amount of dereferincing over the state today, while also keeping
below the limit?
> setup:
> create table a (a int);
> insert into a select x from generate_series(1000000,2000000) x;
>
> bench.sql
> select * from a where a in(1,2,3,4,5,6,7,8,9,10);
>
> f4fb45d15 + reduce_sizeof_hashedsaop_ExprEvalStep.patch
> drowley@amd3990x:~$ pgbench -n -f bench.sql -T 60 -M prepared postgres
> tps = 44.841851 (without initial connection time)
> tps = 44.986472 (without initial connection time)
> tps = 44.944315 (without initial connection time)
>
> f4fb45d15
> drowley@amd3990x:~$ pgbench -n -f bench.sql -T 60 -M prepared postgres
> tps = 44.446127 (without initial connection time)
> tps = 44.614134 (without initial connection time)
> tps = 44.895011 (without initial connection time)
>
> (Patched is ~0.61% faster here)
>
> So, there appears to be no performance regression due to the extra
> indirection. There's maybe even some gains due to the smaller step
> size.
"smaller step size"?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 75+ messages in thread
* Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size David Rowley <dgrowleyml@gmail.com>
2022-06-17 17:21 ` Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
@ 2022-06-28 19:18 ` David Rowley <dgrowleyml@gmail.com>
0 siblings, 0 replies; 75+ messages in thread
From: David Rowley @ 2022-06-28 19:18 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers; Nikita Glukhov <n.gluhov@postgrespro.ru>; Andrew Dunstan <andrew@dunslane.net>
On Sat, 18 Jun 2022 at 05:21, Andres Freund <andres@anarazel.de> wrote:
>
> On 2022-06-17 14:14:54 +1200, David Rowley wrote:
> > So, there appears to be no performance regression due to the extra
> > indirection. There's maybe even some gains due to the smaller step
> > size.
>
> "smaller step size"?
I mean smaller sizeof(ExprEvalStep).
David
^ permalink raw reply [nested|flat] 75+ messages in thread
end of thread, other threads:[~2023-02-24 09:44 UTC | newest]
Thread overview: 75+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-06-16 23:31 PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size Andres Freund <andres@anarazel.de>
2022-06-16 23:37 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 21:34 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-02-22 22:47 ` Andres Freund <andres@anarazel.de>
2023-02-23 18:39 ` Corey Huinker <corey.huinker@gmail.com>
2023-02-23 18:56 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-02-23 19:39 ` Andres Freund <andres@anarazel.de>
2023-02-24 09:44 ` Corey Huinker <corey.huinker@gmail.com>
2023-02-23 19:35 ` Andres Freund <andres@anarazel.de>
2022-06-17 00:16 ` Andres Freund <andres@anarazel.de>
2022-06-17 02:14 ` David Rowley <dgrowleyml@gmail.com>
2022-06-17 03:33 ` Peter Geoghegan <pg@bowt.ie>
2022-06-17 04:53 ` David Rowley <dgrowleyml@gmail.com>
2022-06-17 05:22 ` Andres Freund <andres@anarazel.de>
2022-06-17 06:24 ` Andres Freund <andres@anarazel.de>
2022-06-17 06:54 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-06-17 06:59 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-06-17 07:05 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-06-17 14:33 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 14:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 17:30 ` Andres Freund <andres@anarazel.de>
2022-06-17 17:43 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-06-17 18:02 ` Andres Freund <andres@anarazel.de>
2022-06-17 20:06 ` Andres Freund <andres@anarazel.de>
2022-06-21 21:11 ` Andrew Dunstan <andrew@dunslane.net>
2022-06-21 21:25 ` Andres Freund <andres@anarazel.de>
2022-06-21 21:41 ` Andrew Dunstan <andrew@dunslane.net>
2022-06-23 07:38 ` Michael Paquier <michael@paquier.xyz>
2022-06-24 01:51 ` Andres Freund <andres@anarazel.de>
2022-06-24 14:29 ` Andrew Dunstan <andrew@dunslane.net>
2022-07-05 18:36 ` Andres Freund <andres@anarazel.de>
2022-07-05 19:04 ` Andrew Dunstan <andrew@dunslane.net>
2022-07-08 21:05 ` Andrew Dunstan <andrew@dunslane.net>
2022-07-11 00:29 ` Andres Freund <andres@anarazel.de>
2022-07-15 21:07 ` Andres Freund <andres@anarazel.de>
2022-07-18 19:09 ` Andrew Dunstan <andrew@dunslane.net>
2022-07-19 11:40 ` Amit Langote <amitlangote09@gmail.com>
2022-07-19 15:37 ` Andres Freund <andres@anarazel.de>
2022-07-20 14:09 ` Amit Langote <amitlangote09@gmail.com>
2022-07-21 14:55 ` Amit Langote <amitlangote09@gmail.com>
2022-07-21 15:19 ` Amit Langote <amitlangote09@gmail.com>
2022-07-21 17:12 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-22 03:21 ` Amit Langote <amitlangote09@gmail.com>
2022-07-22 04:13 ` David Rowley <dgrowleyml@gmail.com>
2022-07-22 05:49 ` Amit Langote <amitlangote09@gmail.com>
2022-07-27 08:01 ` Amit Langote <amitlangote09@gmail.com>
2022-07-29 18:27 ` Andrew Dunstan <andrew@dunslane.net>
2022-08-01 23:27 ` Andres Freund <andres@anarazel.de>
2022-08-02 00:39 ` Andres Freund <andres@anarazel.de>
2022-08-02 03:05 ` Amit Langote <amitlangote09@gmail.com>
2022-08-02 15:00 ` Andres Freund <andres@anarazel.de>
2022-08-04 08:01 ` Amit Langote <amitlangote09@gmail.com>
2022-08-05 20:36 ` Andres Freund <andres@anarazel.de>
2022-08-08 07:38 ` Amit Langote <amitlangote09@gmail.com>
2022-08-09 13:59 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 15:03 ` Andrew Dunstan <andrew@dunslane.net>
2022-08-09 18:04 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 18:57 ` Andres Freund <andres@anarazel.de>
2022-08-09 19:59 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-10 13:27 ` Amit Langote <amitlangote09@gmail.com>
2022-08-11 17:08 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-11 17:17 ` Andres Freund <andres@anarazel.de>
2022-08-09 19:17 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-08-09 19:22 ` Andres Freund <andres@anarazel.de>
2022-08-09 19:50 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 20:15 ` Andrew Dunstan <andrew@dunslane.net>
2022-08-09 20:19 ` Jonathan S. Katz <jkatz@postgresql.org>
2022-08-09 20:27 ` Andrew Dunstan <andrew@dunslane.net>
2022-07-05 19:08 ` Andres Freund <andres@anarazel.de>
2022-06-28 23:40 ` David Rowley <dgrowleyml@gmail.com>
2022-07-06 00:32 ` Andres Freund <andres@anarazel.de>
2022-07-06 07:52 ` David Rowley <dgrowleyml@gmail.com>
2022-06-17 12:33 ` James Coleman <jtc331@gmail.com>
2022-06-17 17:21 ` Andres Freund <andres@anarazel.de>
2022-06-28 19:18 ` David Rowley <dgrowleyml@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox