pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedjit and explain nontext
16+ messages / 5 participants
[nested] [flat]
* jit and explain nontext
@ 2020-10-14 19:39 Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 16+ messages in thread
From: Justin Pryzby @ 2020-10-14 19:39 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andres Freund <andres@anarazel.de>; David Rowley <dgrowleyml@gmail.com>
/* don't print information if no JITing happened */
if (!ji || ji->created_functions == 0)
return;
This applies even when (es->format != EXPLAIN_FORMAT_TEXT), which I think is
wrong. Jit use can be determined by cost, so I think jit details should be
shown in non-text format whenever ji!=NULL, even if it's zeros. Arguably, bits
could be omitted if jit_expressions=off or jit_tuple_deforming=off, but I don't
see the point.
--
Justin
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
@ 2020-10-15 01:02 ` David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 16+ messages in thread
From: David Rowley @ 2020-10-15 01:02 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: pgsql-hackers; Andres Freund <andres@anarazel.de>
On Thu, 15 Oct 2020 at 08:39, Justin Pryzby <pryzby@telsasoft.com> wrote:
>
> /* don't print information if no JITing happened */
> if (!ji || ji->created_functions == 0)
> return;
>
> This applies even when (es->format != EXPLAIN_FORMAT_TEXT), which I think is
> wrong. Jit use can be determined by cost, so I think jit details should be
> shown in non-text format whenever ji!=NULL, even if it's zeros. Arguably, bits
> could be omitted if jit_expressions=off or jit_tuple_deforming=off, but I don't
> see the point.
Just for some reference. Some wisdom was shared in [1], which made a
lot of sense to me.
If we apply that, then we just need to decide if displaying any jit
related fields without any jitted expressions is relevant.
I'm a bit undecided.
[1] https://www.postgresql.org/message-id/2276865.1593102811%40sss.pgh.pa.us
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
@ 2020-10-15 01:15 ` Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 16+ messages in thread
From: Tom Lane @ 2020-10-15 01:15 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; pgsql-hackers; Andres Freund <andres@anarazel.de>
David Rowley <dgrowleyml@gmail.com> writes:
> Just for some reference. Some wisdom was shared in [1], which made a
> lot of sense to me.
> If we apply that, then we just need to decide if displaying any jit
> related fields without any jitted expressions is relevant.
Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for
there was that we should print stuff if it's potentially invoked by a
run-time decision, but not if it was excluded at plan time. I'm not
totally clear on whether jitting decisions are fixed by the plan tree
(including its cost values) or if the executor can make different
decisions in different executions of the identical plan tree.
If the latter, then I agree with Justin that this is a bug.
regards, tom lane
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
@ 2020-10-15 01:23 ` David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 1 reply; 16+ messages in thread
From: David Rowley @ 2020-10-15 01:23 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; pgsql-hackers; Andres Freund <andres@anarazel.de>
On Thu, 15 Oct 2020 at 14:15, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> David Rowley <dgrowleyml@gmail.com> writes:
> > Just for some reference. Some wisdom was shared in [1], which made a
> > lot of sense to me.
> > If we apply that, then we just need to decide if displaying any jit
> > related fields without any jitted expressions is relevant.
>
> Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for
> there was that we should print stuff if it's potentially invoked by a
> run-time decision, but not if it was excluded at plan time. I'm not
> totally clear on whether jitting decisions are fixed by the plan tree
> (including its cost values) or if the executor can make different
> decisions in different executions of the identical plan tree.
> If the latter, then I agree with Justin that this is a bug.
As far as I know, the only exception where the executor overwrites the
planner's decision is in nodeValuesscan.c where it turns jit off
because each VALUES will get evaluated just once, which would be a
waste of effort to JIT.
Apart from that the choice is baked in by the planner and set in
PlannedStmt.jitfFlags.
David
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
@ 2020-10-15 01:43 ` Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 16+ messages in thread
From: Justin Pryzby @ 2020-10-15 01:43 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers; Andres Freund <andres@anarazel.de>
On Thu, Oct 15, 2020 at 02:23:01PM +1300, David Rowley wrote:
> On Thu, 15 Oct 2020 at 14:15, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > David Rowley <dgrowleyml@gmail.com> writes:
> > > Just for some reference. Some wisdom was shared in [1], which made a
> > > lot of sense to me.
> > > If we apply that, then we just need to decide if displaying any jit
> > > related fields without any jitted expressions is relevant.
> >
> > Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for
> > there was that we should print stuff if it's potentially invoked by a
> > run-time decision, but not if it was excluded at plan time. I'm not
> > totally clear on whether jitting decisions are fixed by the plan tree
> > (including its cost values) or if the executor can make different
> > decisions in different executions of the identical plan tree.
> > If the latter, then I agree with Justin that this is a bug.
>
> As far as I know, the only exception where the executor overwrites the
> planner's decision is in nodeValuesscan.c where it turns jit off
> because each VALUES will get evaluated just once, which would be a
> waste of effort to JIT.
>
> Apart from that the choice is baked in by the planner and set in
> PlannedStmt.jitfFlags.
What about the GUCs themselves ?
They can change after planning, which means a given execution of a plan might
or might not use jit.
--
Justin
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
@ 2020-10-15 01:51 ` David Rowley <dgrowleyml@gmail.com>
2020-10-15 23:00 ` Re: jit and explain nontext Andres Freund <andres@anarazel.de>
2020-10-17 19:21 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 2 replies; 16+ messages in thread
From: David Rowley @ 2020-10-15 01:51 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers; Andres Freund <andres@anarazel.de>
On Thu, 15 Oct 2020 at 14:43, Justin Pryzby <pryzby@telsasoft.com> wrote:
>
> On Thu, Oct 15, 2020 at 02:23:01PM +1300, David Rowley wrote:
> > On Thu, 15 Oct 2020 at 14:15, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > > Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for
> > > there was that we should print stuff if it's potentially invoked by a
> > > run-time decision, but not if it was excluded at plan time. I'm not
> > > totally clear on whether jitting decisions are fixed by the plan tree
> > > (including its cost values) or if the executor can make different
> > > decisions in different executions of the identical plan tree.
> > > If the latter, then I agree with Justin that this is a bug.
> >
> > As far as I know, the only exception where the executor overwrites the
> > planner's decision is in nodeValuesscan.c where it turns jit off
> > because each VALUES will get evaluated just once, which would be a
> > waste of effort to JIT.
> >
> > Apart from that the choice is baked in by the planner and set in
> > PlannedStmt.jitfFlags.
>
> What about the GUCs themselves ?
>
> They can change after planning, which means a given execution of a plan might
> or might not use jit.
That's a pretty good point. If we do SET enable_sort TO off; then
cached plans are unaffected. That's not the case when someone does
SET jit TO off; as we'll check that in provider_init() during
execution. Although, switching jit back on again works differently.
If the planner saw it was off then switching it on again won't have
existing plans use it. That's slightly weird, but perhaps it was done
that way to ensure there was a hard off switch. You might want to
ensure that to ensure queries don't break if there was some problem
with LLVM libraries.
David
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
@ 2020-10-15 23:00 ` Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 16+ messages in thread
From: Andres Freund @ 2020-10-15 23:00 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
Hi,
On 2020-10-15 14:51:38 +1300, David Rowley wrote:
> That's a pretty good point. If we do SET enable_sort TO off; then
> cached plans are unaffected.
In contrast to that we do however use the current work_mem for
execution, I believe. Which could be fairly nasty, if a plan was made
for a huge work_mem, for example.
> That's not the case when someone does SET jit TO off; as we'll check
> that in provider_init() during execution. Although, switching jit
> back on again works differently. If the planner saw it was off then
> switching it on again won't have existing plans use it. That's
> slightly weird, but perhaps it was done that way to ensure there was a
> hard off switch.
It was motivated by not wanting to just enable JIT for queries that were
prepared within something like SET LOCAL jit=off;PREPARE; RESET
jit;. I'm open to revising it, but that's where it's coming from.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
@ 2020-10-17 19:21 ` Justin Pryzby <pryzby@telsasoft.com>
2020-10-18 22:20 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-11-20 15:56 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
1 sibling, 2 replies; 16+ messages in thread
From: Justin Pryzby @ 2020-10-17 19:21 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Andres Freund <andres@anarazel.de>
On Thu, Oct 15, 2020 at 02:51:38PM +1300, David Rowley wrote:
> On Thu, 15 Oct 2020 at 14:43, Justin Pryzby <pryzby@telsasoft.com> wrote:
> > On Thu, Oct 15, 2020 at 02:23:01PM +1300, David Rowley wrote:
> > > On Thu, 15 Oct 2020 at 14:15, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > > > Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for
> > > > there was that we should print stuff if it's potentially invoked by a
> > > > run-time decision, but not if it was excluded at plan time. I'm not
> > > > totally clear on whether jitting decisions are fixed by the plan tree
> > > > (including its cost values) or if the executor can make different
> > > > decisions in different executions of the identical plan tree.
> > > > If the latter, then I agree with Justin that this is a bug.
> > >
> > > As far as I know, the only exception where the executor overwrites the
> > > planner's decision is in nodeValuesscan.c where it turns jit off
> > > because each VALUES will get evaluated just once, which would be a
> > > waste of effort to JIT.
> > >
> > > Apart from that the choice is baked in by the planner and set in
> > > PlannedStmt.jitfFlags.
> >
> > What about the GUCs themselves ?
> >
> > They can change after planning, which means a given execution of a plan might
> > or might not use jit.
>
> That's a pretty good point.
Added at: https://commitfest.postgresql.org/30/2766/
inline patch 1: src/backend/commands/explain.c (+2/-1 lines) download inline-1.patch
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 41317f1837..7345971507 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -839,7 +839,8 @@ ExplainPrintJIT(ExplainState *es, int jit_flags, JitInstrumentation *ji)
instr_time total_time;
/* don't print information if no JITing happened */
- if (!ji || ji->created_functions == 0)
+ if (!ji || (ji->created_functions == 0 &&
+ es->format == EXPLAIN_FORMAT_TEXT))
return;
/* calculate total time */
--
2.17.0
v1-0001-explain-show-JIT-details-in-non-text-format-even-.patch (text/x-diff, 864B) download
From 237f4d5d0739583683fd7a4d67a6e3d9120aba04 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 17 Oct 2020 14:10:11 -0500
Subject: [PATCH v1] explain: show JIT details in non-text format, even if zero
---
src/backend/commands/explain.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 41317f1837..7345971507 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -839,7 +839,8 @@ ExplainPrintJIT(ExplainState *es, int jit_flags, JitInstrumentation *ji)
instr_time total_time;
/* don't print information if no JITing happened */
- if (!ji || ji->created_functions == 0)
+ if (!ji || (ji->created_functions == 0 &&
+ es->format == EXPLAIN_FORMAT_TEXT))
return;
/* calculate total time */
--
2.17.0
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-17 19:21 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
@ 2020-10-18 22:20 ` David Rowley <dgrowleyml@gmail.com>
1 sibling, 0 replies; 16+ messages in thread
From: David Rowley @ 2020-10-18 22:20 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers; Andres Freund <andres@anarazel.de>
On Sun, 18 Oct 2020 at 08:21, Justin Pryzby <pryzby@telsasoft.com> wrote:
> /* don't print information if no JITing happened */
> - if (!ji || ji->created_functions == 0)
> + if (!ji || (ji->created_functions == 0 &&
> + es->format == EXPLAIN_FORMAT_TEXT))
> return;
Isn't that comment now outdated?
I imagine something more like; /* Only show JIT details when we jitted
something or when in non-text mode */ might be better after making
that code change.
David
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-17 19:21 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
@ 2020-11-20 15:56 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-20 16:16 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
1 sibling, 1 reply; 16+ messages in thread
From: Peter Eisentraut @ 2020-11-20 15:56 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; David Rowley <dgrowleyml@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Andres Freund <andres@anarazel.de>
On 2020-10-17 21:21, Justin Pryzby wrote:
> Added at:https://commitfest.postgresql.org/30/2766/
>
> diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
> index 41317f1837..7345971507 100644
> --- a/src/backend/commands/explain.c
> +++ b/src/backend/commands/explain.c
> @@ -839,7 +839,8 @@ ExplainPrintJIT(ExplainState *es, int jit_flags, JitInstrumentation *ji)
> instr_time total_time;
>
> /* don't print information if no JITing happened */
> - if (!ji || ji->created_functions == 0)
> + if (!ji || (ji->created_functions == 0 &&
> + es->format == EXPLAIN_FORMAT_TEXT))
> return;
>
> /* calculate total time */
Can you show an output example of where this patch makes a difference?
Just from reading the description, I would expect some kind of
additional JIT-related output from something like
EXPLAIN (FORMAT YAML) SELECT 1;
but I don't see anything.
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-17 19:21 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-20 15:56 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
@ 2020-11-20 16:16 ` Justin Pryzby <pryzby@telsasoft.com>
2020-11-21 07:39 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
0 siblings, 1 reply; 16+ messages in thread
From: Justin Pryzby @ 2020-11-20 16:16 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Andres Freund <andres@anarazel.de>
On Fri, Nov 20, 2020 at 04:56:38PM +0100, Peter Eisentraut wrote:
> On 2020-10-17 21:21, Justin Pryzby wrote:
> > Added at:https://commitfest.postgresql.org/30/2766/
> >
> > diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
> > index 41317f1837..7345971507 100644
> > --- a/src/backend/commands/explain.c
> > +++ b/src/backend/commands/explain.c
> > @@ -839,7 +839,8 @@ ExplainPrintJIT(ExplainState *es, int jit_flags, JitInstrumentation *ji)
> > instr_time total_time;
> > /* don't print information if no JITing happened */
> > - if (!ji || ji->created_functions == 0)
> > + if (!ji || (ji->created_functions == 0 &&
> > + es->format == EXPLAIN_FORMAT_TEXT))
> > return;
> > /* calculate total time */
>
> Can you show an output example of where this patch makes a difference? Just
> from reading the description, I would expect some kind of additional
> JIT-related output from something like
>
> EXPLAIN (FORMAT YAML) SELECT 1;
It matters if it was planned with jit but executed without jit.
postgres=# DEALLOCATE p; SET jit=on; SET jit_above_cost=0; prepare p as select from generate_series(1,9); explain(format yaml) execute p; SET jit=off; explain(format yaml) execute p;
Patched shows this for both explains:
JIT: +
Functions: 3 +
Unpatched shows only in the first case.
--
Justin
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-17 19:21 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-20 15:56 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-20 16:16 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
@ 2020-11-21 07:39 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-21 16:26 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 1 reply; 16+ messages in thread
From: Peter Eisentraut @ 2020-11-21 07:39 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Andres Freund <andres@anarazel.de>
On 2020-11-20 17:16, Justin Pryzby wrote:
> It matters if it was planned with jit but executed without jit.
>
> postgres=# DEALLOCATE p; SET jit=on; SET jit_above_cost=0; prepare p as select from generate_series(1,9); explain(format yaml) execute p; SET jit=off; explain(format yaml) execute p;
>
> Patched shows this for both explains:
> JIT: +
> Functions: 3 +
>
> Unpatched shows only in the first case.
In this context, I don't see the point of this change. If you set
jit=off explicitly, then there is no need to clutter the EXPLAIN output
with a bunch of zeroes about JIT.
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-17 19:21 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-20 15:56 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-20 16:16 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-21 07:39 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
@ 2020-11-21 16:26 ` Justin Pryzby <pryzby@telsasoft.com>
2020-11-30 16:59 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 1 reply; 16+ messages in thread
From: Justin Pryzby @ 2020-11-21 16:26 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Andres Freund <andres@anarazel.de>
On Sat, Nov 21, 2020 at 08:39:11AM +0100, Peter Eisentraut wrote:
> On 2020-11-20 17:16, Justin Pryzby wrote:
> > It matters if it was planned with jit but executed without jit.
> >
> > postgres=# DEALLOCATE p; SET jit=on; SET jit_above_cost=0; prepare p as select from generate_series(1,9); explain(format yaml) execute p; SET jit=off; explain(format yaml) execute p;
> >
> > Patched shows this for both explains:
> > JIT: +
> > Functions: 3 +
> >
> > Unpatched shows only in the first case.
>
> In this context, I don't see the point of this change. If you set jit=off
> explicitly, then there is no need to clutter the EXPLAIN output with a bunch
> of zeroes about JIT.
I think the idea is that someone should be able to parse the YAML/XML/other
output by writing something like a['JIT']['Functions'] rather than something
like a.get('JIT',{}).get('Functions',0)
The standard seems to be that parameters that can change during execution
should change the *values* in the non-text output, but the *keys* should not
disappear just because (for example) parallel workers weren't available, or
someone (else) turned off jit. We had discussion about this earlier in the
year:
https://www.postgresql.org/message-id/20200728033622.GC20393@telsasoft.com
(Since it's machine-readable output, Key: 0 is Consistency and Completeness,
not Clutter.)
--
Justin
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-17 19:21 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-20 15:56 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-20 16:16 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-21 07:39 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-21 16:26 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
@ 2020-11-30 16:59 ` Justin Pryzby <pryzby@telsasoft.com>
2021-01-15 19:53 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 16+ messages in thread
From: Justin Pryzby @ 2020-11-30 16:59 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Andres Freund <andres@anarazel.de>
On Sat, Nov 21, 2020 at 10:26:00AM -0600, Justin Pryzby wrote:
> On Sat, Nov 21, 2020 at 08:39:11AM +0100, Peter Eisentraut wrote:
> > On 2020-11-20 17:16, Justin Pryzby wrote:
> > > It matters if it was planned with jit but executed without jit.
> > >
> > > postgres=# DEALLOCATE p; SET jit=on; SET jit_above_cost=0; prepare p as select from generate_series(1,9); explain(format yaml) execute p; SET jit=off; explain(format yaml) execute p;
> > >
> > > Patched shows this for both explains:
> > > JIT: +
> > > Functions: 3 +
> > >
> > > Unpatched shows only in the first case.
> >
> > In this context, I don't see the point of this change. If you set jit=off
> > explicitly, then there is no need to clutter the EXPLAIN output with a bunch
> > of zeroes about JIT.
>
> I think the idea is that someone should be able to parse the YAML/XML/other
> output by writing something like a['JIT']['Functions'] rather than something
> like a.get('JIT',{}).get('Functions',0)
>
> The standard seems to be that parameters that can change during execution
> should change the *values* in the non-text output, but the *keys* should not
> disappear just because (for example) parallel workers weren't available, or
> someone (else) turned off jit. We had discussion about this earlier in the
> year:
> https://www.postgresql.org/message-id/20200728033622.GC20393@telsasoft.com
>
> (Since it's machine-readable output, Key: 0 is Consistency and Completeness,
> not Clutter.)
If there's no interest or agreement in it, we should just close it.
I have no personal need for it, but noticed it in passing.
Attachments:
v2-0001-explain-show-JIT-details-in-non-text-format-even-.patch (text/x-diff, 941B) download
From 8a580c7a90e9854d53c15b69e761161330a6bf87 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 17 Oct 2020 14:10:11 -0500
Subject: [PATCH v2] explain: show JIT details in non-text format, even if zero
---
src/backend/commands/explain.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 43f9b01e83..89caa76801 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -838,8 +838,9 @@ ExplainPrintJIT(ExplainState *es, int jit_flags, JitInstrumentation *ji)
{
instr_time total_time;
- /* don't print information if no JITing happened */
- if (!ji || ji->created_functions == 0)
+ /* don't print information if JITing wasn't done at planning time */
+ if (!ji || (ji->created_functions == 0 &&
+ es->format == EXPLAIN_FORMAT_TEXT))
return;
/* calculate total time */
--
2.17.0
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-17 19:21 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-20 15:56 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-20 16:16 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-21 07:39 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-21 16:26 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-30 16:59 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
@ 2021-01-15 19:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-01-15 20:25 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 1 reply; 16+ messages in thread
From: Tom Lane @ 2021-01-15 19:53 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; David Rowley <dgrowleyml@gmail.com>; pgsql-hackers@postgresql.org, Andres Freund <andres@anarazel.de>
Justin Pryzby <pryzby@telsasoft.com> writes:
> On Sat, Nov 21, 2020 at 10:26:00AM -0600, Justin Pryzby wrote:
>> On Sat, Nov 21, 2020 at 08:39:11AM +0100, Peter Eisentraut wrote:
>>> In this context, I don't see the point of this change. If you set jit=off
>>> explicitly, then there is no need to clutter the EXPLAIN output with a bunch
>>> of zeroes about JIT.
> If there's no interest or agreement in it, we should just close it.
> I have no personal need for it, but noticed it in passing.
I dug around a bit and saw that essentially all of the JIT control
GUCs are consulted only at plan time (cf standard_planner, which
fills PlannedStmt.jitFlags based on the then-active settings).
So the only thing that really counts as a "run time decision"
here is that if you set jit = off between planning and execution,
or if we fail to load the JIT provider at all, then you'll get
no JITting even though the planner expected it to happen.
On balance I agree with Peter's opinion that this isn't worth
changing. I would be for the patch if the executor had a little
more freedom of action, but as things stand there's not much
freedom there.
regards, tom lane
^ permalink raw reply [nested|flat] 16+ messages in thread
* Re: jit and explain nontext
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` Re: jit and explain nontext David Rowley <dgrowleyml@gmail.com>
2020-10-17 19:21 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-20 15:56 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-20 16:16 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-21 07:39 ` Re: jit and explain nontext Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-21 16:26 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-11-30 16:59 ` Re: jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2021-01-15 19:53 ` Re: jit and explain nontext Tom Lane <tgl@sss.pgh.pa.us>
@ 2021-01-15 20:25 ` Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 0 replies; 16+ messages in thread
From: Justin Pryzby @ 2021-01-15 20:25 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; David Rowley <dgrowleyml@gmail.com>; pgsql-hackers@postgresql.org, Andres Freund <andres@anarazel.de>
On Fri, Jan 15, 2021 at 02:53:49PM -0500, Tom Lane wrote:
> On balance I agree with Peter's opinion that this isn't worth
> changing. I would be for the patch if the executor had a little
> more freedom of action, but as things stand there's not much
> freedom there.
Thanks for looking
CF: withdrawn.
--
Justin
^ permalink raw reply [nested|flat] 16+ messages in thread
end of thread, other threads:[~2021-01-15 20:25 UTC | newest]
Thread overview: 16+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-10-14 19:39 jit and explain nontext Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:02 ` David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:15 ` Tom Lane <tgl@sss.pgh.pa.us>
2020-10-15 01:23 ` David Rowley <dgrowleyml@gmail.com>
2020-10-15 01:43 ` Justin Pryzby <pryzby@telsasoft.com>
2020-10-15 01:51 ` David Rowley <dgrowleyml@gmail.com>
2020-10-15 23:00 ` Andres Freund <andres@anarazel.de>
2020-10-17 19:21 ` Justin Pryzby <pryzby@telsasoft.com>
2020-10-18 22:20 ` David Rowley <dgrowleyml@gmail.com>
2020-11-20 15:56 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-20 16:16 ` Justin Pryzby <pryzby@telsasoft.com>
2020-11-21 07:39 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-11-21 16:26 ` Justin Pryzby <pryzby@telsasoft.com>
2020-11-30 16:59 ` Justin Pryzby <pryzby@telsasoft.com>
2021-01-15 19:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-01-15 20:25 ` Justin Pryzby <pryzby@telsasoft.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox