public inbox for [email protected]
help / color / mirror / Atom feedFrom: Richard Guo <[email protected]>
To: Thom Brown <[email protected]>
Cc: pgsql-bugs <[email protected]>
Subject: Re: EXPLAIN (VERBOSE) fails with for JSON_ARRAYAGG/JSON_OBJECTAGG + window function
Date: Mon, 6 Jul 2026 11:17:16 +0900
Message-ID: <CAMbWs4_+Xh6JRTUrxBHhr2-5e+5Y74vne_xR_Dr_=Y0t=5cjgg@mail.gmail.com> (raw)
In-Reply-To: <CAA-aLv6p3tMdY4KJ9wG_DxL1_3MTMCeKj8zUo+1-MctSBmvfdw@mail.gmail.com>
References: <CAA-aLv5QYTaMOk=Qhv6cgwceeHETZV8YJvWZ_rH+yVZCuchATA@mail.gmail.com>
<CAMbWs4_0_DmnVc=oHbkqOCMnwwdM5GUDZzioPA+o4-WTsNnYSQ@mail.gmail.com>
<CAMbWs4-b2By5XFEn-_ZJKgN4-8SBhAoRZJrt4gwmM2ctOQzAyw@mail.gmail.com>
<CAA-aLv6p3tMdY4KJ9wG_DxL1_3MTMCeKj8zUo+1-MctSBmvfdw@mail.gmail.com>
On Fri, Jul 3, 2026 at 5:36 PM Thom Brown <[email protected]> wrote:
> Thanks for taking a look at this. It's unfortunate that reconstructing
> the syntax is problematic, because in the case of the original query
> that tripped on this bug, I lose the following from my original query:
> ...
> I guess the relevant information can be gleaned from this, but not for
> JSON_ARRAYAGG(i RETURNING text) because we get the same output whether
> we're returning text or json.
Yeah, the v1 patch is lossy, and the lost information cannot be
recovered from the plan at all.
So I'm switching back to the resolve_special_varno() reconstruction I
had tried and set aside. My earlier objection was that it makes the
WindowAgg look like it computes a JSON aggregate when it only passes
through a value computed below. But the lower node still prints
jsonb_agg_strict(name), so the plan does show where the aggregate is
computed, and I think that is a smaller cost than losing the
information outright. Also, partial aggregation already deparses this
way. The combining aggregate's argument is a Var referencing the
partial aggregate's output, and get_agg_expr() resolves it back with
resolve_special_varno() to reprint the aggregate at the finalizing
node.
I still don't want to hack the planner to keep the JsonConstructorExpr
with its Aggref in make_window_input_target(), because I still think
that that is too invasive and changes which node evaluates the
wrapper.
Hence, I end up with the attached v2 patch.
- Richard
Attachments:
[application/octet-stream] v2-0001-Fix-EXPLAIN-failure-when-deparsing-SQL-JSON-aggre.patch (11.2K, ../CAMbWs4_+Xh6JRTUrxBHhr2-5e+5Y74vne_xR_Dr_=Y0t=5cjgg@mail.gmail.com/2-v2-0001-Fix-EXPLAIN-failure-when-deparsing-SQL-JSON-aggre.patch)
download | inline diff:
From 1cc7e77c4bf6bede72fe5756390ac86e77092a7b Mon Sep 17 00:00:00 2001
From: Richard Guo <[email protected]>
Date: Fri, 3 Jul 2026 10:52:29 +0900
Subject: [PATCH v2] Fix EXPLAIN failure when deparsing SQL/JSON aggregates
If an expression containing an aggregate is evaluated above the plan
node that computes the aggregate, as happens with window functions or
with expressions postponed to above the final sort, setrefs.c replaces
the Aggref or WindowFunc with a Var referencing the lower node's
output. For SQL/JSON aggregates such as JSON_ARRAYAGG, deparsing the
containing JsonConstructorExpr then failed with "invalid
JsonConstructorExpr underlying node type", since
get_json_agg_constructor() did not expect a Var there.
Fix by resolving the Var back to the underlying Aggref or WindowFunc
and deparsing the constructor as if the aggregate were computed at the
current node. The JsonConstructorExpr retains the RETURNING clause
and the ABSENT/NULL ON NULL and WITH UNIQUE options, and the arguments
come from the resolved aggregate, so the original JSON aggregate
syntax is reproduced in full. This mirrors how get_agg_expr() already
looks through such a Var when deparsing a combining aggregate.
---
src/backend/utils/adt/ruleutils.c | 32 ++++++++
src/test/regress/expected/sqljson.out | 101 ++++++++++++++++++++++++++
src/test/regress/sql/sqljson.sql | 41 +++++++++++
3 files changed, 174 insertions(+)
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 88de5c0481c..819631781c0 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -512,6 +512,8 @@ static void get_json_agg_constructor(JsonConstructorExpr *ctor,
deparse_context *context,
const char *funcname,
bool is_json_objectagg);
+static void get_json_agg_constructor_expr(Node *node, deparse_context *context,
+ void *callback_arg);
static void simple_quote_literal(StringInfo buf, const char *val);
static void get_sublink_expr(SubLink *sublink, deparse_context *context);
static void get_tablefunc(TableFunc *tf, deparse_context *context,
@@ -12389,11 +12391,41 @@ get_json_agg_constructor(JsonConstructorExpr *ctor, deparse_context *context,
get_windowfunc_expr_helper((WindowFunc *) ctor->func, context,
funcname, options.data,
is_json_objectagg);
+ else if (IsA(ctor->func, Var))
+ {
+ /*
+ * If the aggregate is computed by a lower plan node, setrefs.c will
+ * have replaced the Aggref or WindowFunc with a Var referencing that
+ * node's output. Chase the Var back to it so we can still print the
+ * original JSON aggregate syntax. This only happens in EXPLAIN.
+ */
+ resolve_special_varno((Node *) ctor->func, context,
+ get_json_agg_constructor_expr, ctor);
+ }
else
elog(ERROR, "invalid JsonConstructorExpr underlying node type: %d",
nodeTag(ctor->func));
}
+/*
+ * Deparse a JsonConstructorExpr whose aggregate is computed by a lower plan
+ * node; resolve_special_varno has located the underlying Aggref/WindowFunc.
+ */
+static void
+get_json_agg_constructor_expr(Node *node, deparse_context *context,
+ void *callback_arg)
+{
+ JsonConstructorExpr ctor;
+
+ if (!IsA(node, Aggref) && !IsA(node, WindowFunc))
+ elog(ERROR, "JSON aggregate constructor does not point to an Aggref or WindowFunc");
+
+ /* Flat copy suffices; we only replace func. */
+ ctor = *(JsonConstructorExpr *) callback_arg;
+ ctor.func = (Expr *) node;
+ get_json_constructor(&ctor, context, false);
+}
+
/*
* simple_quote_literal - Format a string as a SQL literal, append to buf
*/
diff --git a/src/test/regress/expected/sqljson.out b/src/test/regress/expected/sqljson.out
index 0f337bda325..091a0b98574 100644
--- a/src/test/regress/expected/sqljson.out
+++ b/src/test/regress/expected/sqljson.out
@@ -1757,3 +1757,104 @@ SELECT JSON_OBJECT('a': JSON_OBJECTAGG('b': 1 RETURNING text) FORMAT JSON);
(1 row)
DROP FUNCTION volatile_one, stable_one;
+-- Test deparsing of JSON aggregates that are computed below a WindowAgg
+-- node.
+EXPLAIN (VERBOSE, COSTS OFF)
+SELECT i % 2 AS g,
+ JSON_ARRAYAGG(i ORDER BY i RETURNING jsonb) AS ja,
+ JSON_ARRAYAGG(i ORDER BY i RETURNING text) AS ja_text,
+ JSON_ARRAYAGG(i ORDER BY i NULL ON NULL RETURNING jsonb) AS ja_null,
+ JSON_OBJECTAGG(i: i ABSENT ON NULL RETURNING jsonb) AS jo_absent,
+ JSON_OBJECTAGG(i: i WITH UNIQUE RETURNING jsonb) AS jo_unique,
+ row_number() OVER (ORDER BY i % 2) AS rn
+FROM generate_series(1, 3) i
+GROUP BY i % 2;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ WindowAgg
+ Output: ((i % 2)), JSON_ARRAYAGG(i ORDER BY i RETURNING jsonb), JSON_ARRAYAGG(i ORDER BY i RETURNING text), JSON_ARRAYAGG(i ORDER BY i NULL ON NULL RETURNING jsonb), JSON_OBJECTAGG(i : i ABSENT ON NULL RETURNING jsonb), JSON_OBJECTAGG(i : i WITH UNIQUE KEYS RETURNING jsonb), row_number() OVER w1
+ Window: w1 AS (ORDER BY ((i.i % 2)) ROWS UNBOUNDED PRECEDING)
+ -> GroupAggregate
+ Output: ((i % 2)), jsonb_agg_strict(i ORDER BY i), json_agg_strict(i ORDER BY i), jsonb_agg(i ORDER BY i), jsonb_object_agg_strict(i, i), jsonb_object_agg_unique(i, i)
+ Group Key: ((i.i % 2))
+ -> Sort
+ Output: ((i % 2)), i
+ Sort Key: ((i.i % 2)), i.i
+ -> Function Scan on pg_catalog.generate_series i
+ Output: (i % 2), i
+ Function Call: generate_series(1, 3)
+(12 rows)
+
+SELECT i % 2 AS g,
+ JSON_ARRAYAGG(i ORDER BY i RETURNING jsonb) AS ja,
+ JSON_ARRAYAGG(i ORDER BY i RETURNING text) AS ja_text,
+ JSON_ARRAYAGG(i ORDER BY i NULL ON NULL RETURNING jsonb) AS ja_null,
+ JSON_OBJECTAGG(i: i ABSENT ON NULL RETURNING jsonb) AS jo_absent,
+ JSON_OBJECTAGG(i: i WITH UNIQUE RETURNING jsonb) AS jo_unique,
+ row_number() OVER (ORDER BY i % 2) AS rn
+FROM generate_series(1, 3) i
+GROUP BY i % 2;
+ g | ja | ja_text | ja_null | jo_absent | jo_unique | rn
+---+--------+---------+---------+------------------+------------------+----
+ 0 | [2] | [2] | [2] | {"2": 2} | {"2": 2} | 1
+ 1 | [1, 3] | [1, 3] | [1, 3] | {"1": 1, "3": 3} | {"1": 1, "3": 3} | 2
+(2 rows)
+
+-- The same, but with the JSON aggregate used as a window function that is
+-- computed below another WindowAgg node.
+EXPLAIN (VERBOSE, COSTS OFF)
+SELECT JSON_ARRAYAGG(i NULL ON NULL RETURNING jsonb) OVER (ORDER BY i DESC) AS ja,
+ row_number() OVER (ORDER BY i) AS rn
+FROM generate_series(1, 3) i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------
+ WindowAgg
+ Output: JSON_ARRAYAGG(i NULL ON NULL RETURNING jsonb) OVER w1, row_number() OVER w2, i
+ Window: w2 AS (ORDER BY i.i ROWS UNBOUNDED PRECEDING)
+ -> Sort
+ Output: i, (jsonb_agg(i) OVER w1)
+ Sort Key: i.i
+ -> WindowAgg
+ Output: i, jsonb_agg(i) OVER w1
+ Window: w1 AS (ORDER BY i.i)
+ -> Sort
+ Output: i
+ Sort Key: i.i DESC
+ -> Function Scan on pg_catalog.generate_series i
+ Output: i
+ Function Call: generate_series(1, 3)
+(15 rows)
+
+SELECT JSON_ARRAYAGG(i NULL ON NULL RETURNING jsonb) OVER (ORDER BY i DESC) AS ja,
+ row_number() OVER (ORDER BY i) AS rn
+FROM generate_series(1, 3) i;
+ ja | rn
+-----------+----
+ [3, 2, 1] | 1
+ [3, 2] | 2
+ [3] | 3
+(3 rows)
+
+-- The same, but with the expression containing the JSON aggregate postponed
+-- to above the final sort due to being volatile.
+EXPLAIN (VERBOSE, COSTS OFF)
+SELECT i % 2 AS g,
+ JSON_ARRAYAGG(i RETURNING text) || random()::text AS ja
+FROM generate_series(1, 3) i
+GROUP BY i % 2
+ORDER BY count(*);
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Result
+ Output: ((i % 2)), (JSON_ARRAYAGG(i RETURNING text) || (random())::text), (count(*))
+ -> Sort
+ Output: ((i % 2)), (count(*)), (json_agg_strict(i))
+ Sort Key: (count(*))
+ -> HashAggregate
+ Output: ((i % 2)), count(*), json_agg_strict(i)
+ Group Key: (i.i % 2)
+ -> Function Scan on pg_catalog.generate_series i
+ Output: (i % 2), i
+ Function Call: generate_series(1, 3)
+(11 rows)
+
diff --git a/src/test/regress/sql/sqljson.sql b/src/test/regress/sql/sqljson.sql
index a68747733a1..2550da15c45 100644
--- a/src/test/regress/sql/sqljson.sql
+++ b/src/test/regress/sql/sqljson.sql
@@ -706,3 +706,44 @@ SELECT JSON_OBJECT('a': JSON_OBJECTAGG('b': stable_one() RETURNING text) FORMAT
EXPLAIN (VERBOSE, COSTS OFF) SELECT JSON_OBJECT('a': JSON_OBJECTAGG('b': 1 RETURNING text) FORMAT JSON);
SELECT JSON_OBJECT('a': JSON_OBJECTAGG('b': 1 RETURNING text) FORMAT JSON);
DROP FUNCTION volatile_one, stable_one;
+
+-- Test deparsing of JSON aggregates that are computed below a WindowAgg
+-- node.
+EXPLAIN (VERBOSE, COSTS OFF)
+SELECT i % 2 AS g,
+ JSON_ARRAYAGG(i ORDER BY i RETURNING jsonb) AS ja,
+ JSON_ARRAYAGG(i ORDER BY i RETURNING text) AS ja_text,
+ JSON_ARRAYAGG(i ORDER BY i NULL ON NULL RETURNING jsonb) AS ja_null,
+ JSON_OBJECTAGG(i: i ABSENT ON NULL RETURNING jsonb) AS jo_absent,
+ JSON_OBJECTAGG(i: i WITH UNIQUE RETURNING jsonb) AS jo_unique,
+ row_number() OVER (ORDER BY i % 2) AS rn
+FROM generate_series(1, 3) i
+GROUP BY i % 2;
+SELECT i % 2 AS g,
+ JSON_ARRAYAGG(i ORDER BY i RETURNING jsonb) AS ja,
+ JSON_ARRAYAGG(i ORDER BY i RETURNING text) AS ja_text,
+ JSON_ARRAYAGG(i ORDER BY i NULL ON NULL RETURNING jsonb) AS ja_null,
+ JSON_OBJECTAGG(i: i ABSENT ON NULL RETURNING jsonb) AS jo_absent,
+ JSON_OBJECTAGG(i: i WITH UNIQUE RETURNING jsonb) AS jo_unique,
+ row_number() OVER (ORDER BY i % 2) AS rn
+FROM generate_series(1, 3) i
+GROUP BY i % 2;
+
+-- The same, but with the JSON aggregate used as a window function that is
+-- computed below another WindowAgg node.
+EXPLAIN (VERBOSE, COSTS OFF)
+SELECT JSON_ARRAYAGG(i NULL ON NULL RETURNING jsonb) OVER (ORDER BY i DESC) AS ja,
+ row_number() OVER (ORDER BY i) AS rn
+FROM generate_series(1, 3) i;
+SELECT JSON_ARRAYAGG(i NULL ON NULL RETURNING jsonb) OVER (ORDER BY i DESC) AS ja,
+ row_number() OVER (ORDER BY i) AS rn
+FROM generate_series(1, 3) i;
+
+-- The same, but with the expression containing the JSON aggregate postponed
+-- to above the final sort due to being volatile.
+EXPLAIN (VERBOSE, COSTS OFF)
+SELECT i % 2 AS g,
+ JSON_ARRAYAGG(i RETURNING text) || random()::text AS ja
+FROM generate_series(1, 3) i
+GROUP BY i % 2
+ORDER BY count(*);
--
2.39.5 (Apple Git-154)
view thread (9+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: EXPLAIN (VERBOSE) fails with for JSON_ARRAYAGG/JSON_OBJECTAGG + window function
In-Reply-To: <CAMbWs4_+Xh6JRTUrxBHhr2-5e+5Y74vne_xR_Dr_=Y0t=5cjgg@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox