agora inbox for [email protected]  

help / color / mirror / Atom feed

[PATCH v4 3/3] Better express platform requirements in s_lock.h.
282+ messages / 2 participants
[nested] [flat]

* [PATCH v4 3/3] Better express platform requirements in s_lock.h.
@ 2026-05-07 20:32  Nathan Bossart <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Nathan Bossart @ 2026-05-07 20:32 UTC (permalink / raw)

---
 src/include/storage/s_lock.h | 56 ++++++++++++++++++++----------------
 1 file changed, 32 insertions(+), 24 deletions(-)

diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h
index fb872edd2f0..c4369ee4ff6 100644
--- a/src/include/storage/s_lock.h
+++ b/src/include/storage/s_lock.h
@@ -44,23 +44,16 @@
  *		atomic test-and-set only when it appears free.
  *
  *	TAS() and TAS_SPIN() are NOT part of the API, and should never be called
- *	directly.
- *
- *	CAUTION: on some platforms TAS() and/or TAS_SPIN() may sometimes report
- *	failure to acquire a lock even when the lock is not locked.  For example,
- *	on Alpha TAS() will "fail" if interrupted.  Therefore a retry loop must
- *	always be used, even if you are certain the lock is free.
+ *	directly.  If a platform-specific TAS() is defined, the platform must
+ *	_not_ define its own S_LOCK().  Conversely, if a platform-specific
+ *	S_LOCK() is defined, the platform must _not_ define its own TAS(), but
+ *	it does need to define its own TAS_SPIN().  Currently, all supported
+ *	platforms define TAS() and use the default S_LOCK() implementation, so
+ *	that is probably a good place to start if adding a new one.
  *
  *	It is the responsibility of these macros to make sure that the compiler
  *	does not re-order accesses to shared memory to precede the actual lock
- *	acquisition, or follow the lock release.  Prior to PostgreSQL 9.5, this
- *	was the caller's responsibility, which meant that callers had to use
- *	volatile-qualified pointers to refer to both the spinlock itself and the
- *	shared data being accessed within the spinlocked critical section.  This
- *	was notationally awkward, easy to forget (and thus error-prone), and
- *	prevented some useful compiler optimizations.  For these reasons, we
- *	now require that the macros themselves prevent compiler re-ordering,
- *	so that the caller doesn't need to take special precautions.
+ *	acquisition, or follow the lock release.
  *
  *	On platforms with weak memory ordering, the TAS(), TAS_SPIN(), and
  *	S_UNLOCK() macros must further include hardware-level memory fence
@@ -72,7 +65,7 @@
  *
  *	On most supported platforms, TAS() uses a tas() function written
  *	in assembly language to execute a hardware atomic-test-and-set
- *	instruction.  Equivalent OS-supplied mutex routines could be used too.
+ *	instruction.  Equivalent compiler intrinsics are another popular option.
  *
  *
  * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
@@ -642,20 +635,25 @@ spin_delay(void)
 #endif	/* !defined(TAS) */
 
 
-/* Blow up if we didn't have any way to do spinlocks */
-#ifndef TAS
-#error PostgreSQL does not have spinlock support on this platform.  Please report this to [email protected].
-#endif
-
-
 /*
  * Default Definitions - override these above as needed.
  */
 
-#if !defined(S_LOCK)
+/*
+ * Make sure S_LOCK is defined, either explicitly for the platform or via a TAS
+ * macro for the platform.  Exactly one of either S_LOCK or TAS should be
+ * defined for a supported platform at this point in the file.
+ */
+#if defined(S_LOCK)
+#if defined(TAS)
+#error Both TAS and S_LOCK defined on this platform.  Please report this to [email protected].
+#endif
+#elif defined(TAS)
 #define S_LOCK(lock) \
 	(TAS(lock) ? s_lock((lock), __FILE__, __LINE__, __func__) : 0)
-#endif	 /* S_LOCK */
+#else
+#error Neither TAS nor S_LOCK defined on this platform.  Please report this to [email protected].
+#endif
 
 #if !defined(S_UNLOCK)
 /*
@@ -687,9 +685,19 @@ extern void s_unlock(volatile slock_t *lock);
 #define SPIN_DELAY()	((void) 0)
 #endif	 /* SPIN_DELAY */
 
+/*
+ * We can only define TAS_SPIN if TAS was defined.  Otherwise, the platform
+ * defined its own S_LOCK without TAS, and therefore is responsible for
+ * defining its own TAS_SPIN as well.  (Note that we currently do not have any
+ * platforms that don't define TAS.)
+ */
 #if !defined(TAS_SPIN)
+#if defined(TAS)
 #define TAS_SPIN(lock)	TAS(lock)
-#endif	 /* TAS_SPIN */
+#else
+#error Neither TAS nor TAS_SPIN defined on this platform.  Please report this to [email protected].
+#endif	 /* TAS */
+#endif	 /* ! TAS_SPIN */
 
 
 /*
-- 
2.50.1 (Apple Git-155)


--JmXY/Yn6swRVDyih--





^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index a758e1a..d08b122 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -108,6 +108,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -200,6 +202,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -568,6 +571,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -921,16 +925,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -943,8 +968,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1374,6 +1399,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg17.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 9b7187e..21d21ea 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -115,6 +115,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -284,6 +286,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -703,6 +706,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -983,16 +987,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1005,8 +1030,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1924,6 +1949,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 8cf6ecf..0bbad25 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index acb508c..1d867d8 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg18.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index eedc5c8..c4d5cb1 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -282,6 +284,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -701,6 +704,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -981,16 +985,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1003,8 +1028,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1923,6 +1948,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 4bcd4e9..03fbd97 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 3e8929a..db2a21d 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc--






^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 6cc2acb..7dfc311 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -113,6 +113,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -310,6 +312,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -739,6 +742,7 @@ executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
 	cxt.lastGeneratedObjectId = 1 + countVars(vars);
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenceOfErrors(&cxt) && !result)
@@ -1017,16 +1021,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -1039,8 +1064,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1978,6 +2003,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 81efebc..21f1362 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -785,6 +785,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index c1f4ab5..92abe13 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -150,6 +150,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg15.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread

* [PATCH] Collapse consecutive .** accessors for jsonpath exists queries
@ 2026-06-18 20:30  Andrey Rachitskiy <[email protected]>
  0 siblings, 0 replies; 282+ messages in thread

From: Andrey Rachitskiy @ 2026-06-18 20:30 UTC (permalink / raw)

When a jsonpath expression contains multiple consecutive .** (jpiAny)
accessors, each one triggers a full subtree traversal in executeAnyItem().
k consecutive .** operators can degrade performance to O(N^k) on a document
with N nodes, even though a chain like $.**.**.** is redundant for existence
semantics and equivalent to a single $.** with merged level bounds.

This was reported as a performance problem when expressions such as
$.**.**.**.**.* are evaluated with @? against deeply nested JSON: the same
query without the trailing .* completes in sub-millisecond time, while the
form with redundant .** segments can run for many minutes.  A similar pattern
in strict mode can also provoke very large memory allocation attempts.

Collapse consecutive jpiAny nodes at execution time for existence queries
(@? and jsonb_path_exists), merging their level bounds and performing a
single executeAnyItem() pass.

Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Backpatch-through: 15
---
 src/backend/utils/adt/jsonpath_exec.c     | 47 +++++++++++++++++++---
 .../src/test/regress/expected/jsonb_jsonpath.out   | 40 ++++++++++++++++++
 src/test/regress/sql/jsonb_jsonpath.sql   | 11 +++++
 3 files changed, 93 insertions(+), 5 deletions(-)

diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ba9bbb6..74fc499 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -107,6 +107,8 @@ typedef struct JsonPathExecContext
 										 * ignored */
 	bool		throwErrors;	/* with "false" all suppressible errors are
 								 * suppressed */
+	bool		existsOnly;		/* @? / jsonb_path_exists: may collapse
+							 * redundant consecutive .** accessors */
 	bool		useTz;
 } JsonPathExecContext;
 
@@ -199,6 +201,7 @@ static JsonPathExecResult executeAnyItem(JsonPathExecContext *cxt,
 										 JsonPathItem *jsp, JsonbContainer *jbc, JsonValueList *found,
 										 uint32 level, uint32 first, uint32 last,
 										 bool ignoreStructuralErrors, bool unwrapNext);
+static uint32 mergeAnyBound(uint32 a, uint32 b);
 static JsonPathBool executePredicate(JsonPathExecContext *cxt,
 									 JsonPathItem *pred, JsonPathItem *larg, JsonPathItem *rarg,
 									 JsonbValue *jb, bool unwrapRightArg,
@@ -567,6 +570,7 @@ executeJsonPath(JsonPath *path, Jsonb *vars, Jsonb *json, bool throwErrors,
 	cxt.lastGeneratedObjectId = vars ? 2 : 1;
 	cxt.innermostArraySize = -1;
 	cxt.throwErrors = throwErrors;
+	cxt.existsOnly = (result == NULL);
 	cxt.useTz = useTz;
 
 	if (jspStrictAbsenseOfErrors(&cxt) && !result)
@@ -920,16 +924,37 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 
 		case jpiAny:
 			{
-				bool		hasNext = jspGetNext(jsp, &elem);
+				JsonPathItem elem;
+				bool		hasNext;
+				uint32		first;
+				uint32		last;
+
+				hasNext = jspGetNext(jsp, &elem);
+				first = jsp->content.anybounds.first;
+				last = jsp->content.anybounds.last;
+
+				/*
+				 * Consecutive .** accessors are redundant for existence
+				 * queries and multiply traversal cost.
+				 */
+				if (cxt->existsOnly)
+				{
+					while (hasNext && elem.type == jpiAny)
+					{
+						first = mergeAnyBound(first, elem.content.anybounds.first);
+						last = mergeAnyBound(last, elem.content.anybounds.last);
+						hasNext = jspGetNext(&elem, &elem);
+					}
+				}
 
 				/* first try without any intermediate steps */
-				if (jsp->content.anybounds.first == 0)
+				if (first == 0)
 				{
 					bool		savedIgnoreStructuralErrors;
 
 					savedIgnoreStructuralErrors = cxt->ignoreStructuralErrors;
 					cxt->ignoreStructuralErrors = true;
-					res = executeNextItem(cxt, jsp, &elem,
+					res = executeNextItem(cxt, jsp, hasNext ? &elem : NULL,
 										  jb, found, true);
 					cxt->ignoreStructuralErrors = savedIgnoreStructuralErrors;
 
@@ -942,8 +967,8 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp,
 						(cxt, hasNext ? &elem : NULL,
 						 jb->val.binary.data, found,
 						 1,
-						 jsp->content.anybounds.first,
-						 jsp->content.anybounds.last,
+						 first,
+						 last,
 						 true, jspAutoUnwrap(cxt));
 				break;
 			}
@@ -1373,6 +1398,18 @@ executeNestedBoolItem(JsonPathExecContext *cxt, JsonPathItem *jsp,
 	return res;
 }
 
+
+/*
+ * Merge level bounds of two consecutive .** accessors.
+ */
+static uint32
+mergeAnyBound(uint32 a, uint32 b)
+{
+	if (a == PG_UINT32_MAX || b == PG_UINT32_MAX)
+		return PG_UINT32_MAX;
+	return a + b;
+}
+
 /*
  * Implementation of several jsonpath nodes:
  *  - jpiAny (.** accessor),
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index eafb421..5e1bd19 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -792,6 +792,46 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
  t
 (1 row)
 
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+ ?column? 
+----------
+ t
+(1 row)
+
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+ ?column? 
+----------
+ f
+(1 row)
+
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+ jsonb_path_exists 
+-------------------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+ ?column? 
+----------
+ t
+(1 row)
+
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
  jsonb_path_query 
 ------------------
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index 8163fc6..4c62fe2 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -155,6 +155,17 @@ select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{0 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to last}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{1 to 2}.b ? ( @ > 0)';
 select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**{2 to 3}.b ? ( @ > 0)';
+-- Redundant consecutive .** accessors are collapsed for existence queries
+-- (@? and jsonb_path_exists), avoiding multiplicative traversal cost.
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 0)';
+select jsonb '{"a": {"c": {"b": 1}}}' @? '$.**.**.b ? (@ > 99)';
+select jsonb_path_exists('{"a": {"c": {"b": 1}}}', '$.**.**.b ? (@ > 0)');
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'lax $.**.**.**.**';
+select ('[' || repeat('[', 50) || '0' || repeat(']', 50) || ']')::jsonb
+	@? 'strict $.**.**.**';
+
 
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.x))');
 select jsonb_path_query('{"g": {"x": 2}}', '$.g ? (exists (@.y))');
-- 
2.47.3

--MP_/BlgiDE/t55y8Vb7V2uvNybc
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-collapse-consecutive-jsonpath-any-pg16.patch



^ permalink  raw  reply  [nested|flat] 282+ messages in thread