public inbox for [email protected]
help / color / mirror / Atom feedBUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
13+ messages / 5 participants
[nested] [flat]
* BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-04-17 11:20 PG Bug reporting form <[email protected]>
0 siblings, 1 reply; 13+ messages in thread
From: PG Bug reporting form @ 2026-04-17 11:20 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]
The following bug has been logged on the website:
Bug reference: 19458
Logged by: Andrey Rachitskiy
Email address: [email protected]
PostgreSQL version: 14.22
Operating system: Debian GNU/Linux 12 (bookworm)
Description:
Description:
During fuzzing of the jsonb_path_exists_opr (operator jsonb @? jsonpath, a
two-argument version of jsonb_path_exists()), a pathological query was
discovered that causes uncontrolled memory consumption, leading to OOM
Killer on PostgreSQL versions REL_14/15/16_STABLE.
On versions 17 and 18, the same query returns a proper error instead of
crashing the server.
This bug was found using AFL++ as a fuzzer and LibBlobStamper as a tool for
creating syntactically correct arguments.
Reproduction:
Execute the following query:
```sql
select '[3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
13558284848669739, 3472328296227668016, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328295419228208, 3472328296227680304,
3528904766546522246, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296328343600,
3472328296227680304, 3472328296227680299, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3470920921344127024, 3906362710315511856,
3472328296228075062, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472334893297446960, 3472328090069248816,
13511005849006128, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
3472328296227680304, 3472328296227680304, 3472328296227680304,
13563782407139376, 4337019423877509168]'::jsonb @? '(-$?(0 <
($"〰〭〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰" - $?(0 < $"〰〰〰〰〰〰〰〰〰〰〰〰〰〰" - $?(0 + $ <
$"㘰〰㘶〰")."〰〰〰〰")."〰〰〰〰〰〰〰〰") - 0?(+$ < $"〰
〰〰〰")."ほ〰〰㘰")."〰〰〰〶〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰")'::jsonpath;
```
Expected result:
The query should return an error, as happens on versions 17 and 18:
ERROR: could not find jsonpath variable "〰〭〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰"
Actual result (14, 15, 16):
- Memory consumption grows until the kernel kills the postgres process via
OOM Killer
- Client loses connection:
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Kernel log:
516294.487767] Out of memory: Killed process 1135405 (postgres)
total-vm:13521932kB, anon-rss:9170792kB, file-rss:92kB, shmem-rss:1848kB,
UID:1002 pgtables:26176kB oom_score_adj:0
--
Regards,
Andrey Rachitskiy
Postgres Professional
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-04-20 13:38 Andrey Rachitskiy <[email protected]>
parent: PG Bug reporting form <[email protected]>
0 siblings, 1 reply; 13+ messages in thread
From: Andrey Rachitskiy @ 2026-04-20 13:38 UTC (permalink / raw)
To: [email protected]; [email protected]; +Cc: [email protected]
I propose a targeted backpatch for REL_14/15/16 in jsonpath_exec.c to align
missing variable handling with newer branches and prevent pathological
memory growth on malformed/hostile jsonpath expressions.
Why this is not a full backport
This is intentionally not a full backport of the REL_17 jsonpath executor
refactoring (callbacks for variable access, broader executor integration,
etc.).
Why this still fixes the reported issue
On REL_14/15/16, with current behavior (missing vars -> null), some crafted
jsonpath expressions continue deep evaluation and can consume very large
memory, leading to OOM.
With this patch, execution fails early on undefined variable, matching
REL_17 behavior for this case and avoiding runaway memory use.
—
Regards,
Andrey Rachitskiy
Postgres Professional
пт, 17 апр. 2026 г. в 16:21, PG Bug reporting form <[email protected]>:
> The following bug has been logged on the website:
>
> Bug reference: 19458
> Logged by: Andrey Rachitskiy
> Email address: [email protected]
> PostgreSQL version: 14.22
> Operating system: Debian GNU/Linux 12 (bookworm)
> Description:
>
> Description:
> During fuzzing of the jsonb_path_exists_opr (operator jsonb @? jsonpath, a
> two-argument version of jsonb_path_exists()), a pathological query was
> discovered that causes uncontrolled memory consumption, leading to OOM
> Killer on PostgreSQL versions REL_14/15/16_STABLE.
> On versions 17 and 18, the same query returns a proper error instead of
> crashing the server.
> This bug was found using AFL++ as a fuzzer and LibBlobStamper as a tool for
> creating syntactically correct arguments.
>
> Reproduction:
> Execute the following query:
> ```sql
> select '[3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 13558284848669739, 3472328296227668016, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328295419228208, 3472328296227680304,
> 3528904766546522246, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296328343600,
> 3472328296227680304, 3472328296227680299, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3470920921344127024, 3906362710315511856,
> 3472328296228075062, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472334893297446960, 3472328090069248816,
> 13511005849006128, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 3472328296227680304, 3472328296227680304, 3472328296227680304,
> 13563782407139376, 4337019423877509168]'::jsonb @? '(-$?(0 <
> ($"〰〭〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰" - $?(0 < $"〰〰〰〰〰〰〰〰〰〰〰〰〰〰" - $?(0 + $ <
> $"㘰〰㘶〰")."〰〰〰〰")."〰〰〰〰〰〰〰〰") - 0?(+$ < $"〰
>
> 〰〰〰")."ほ〰〰㘰")."〰〰〰〶〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰")'::jsonpath;
> ```
>
> Expected result:
> The query should return an error, as happens on versions 17 and 18:
> ERROR: could not find jsonpath variable "〰〭〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰"
>
> Actual result (14, 15, 16):
> - Memory consumption grows until the kernel kills the postgres process
> via
> OOM Killer
> - Client loses connection:
> server closed the connection unexpectedly
> This probably means the server terminated abnormally
> before or while processing the request.
>
> Kernel log:
> 516294.487767] Out of memory: Killed process 1135405 (postgres)
> total-vm:13521932kB, anon-rss:9170792kB, file-rss:92kB, shmem-rss:1848kB,
> UID:1002 pgtables:26176kB oom_score_adj:0
>
> --
> Regards,
> Andrey Rachitskiy
> Postgres Professional
>
>
>
Attachments:
[application/octet-stream] jsonpath-fix-null-comparison.patch (768B, ../../CAB8bMit1HvJsAasUYwmq+82Oa3zQhJyvsHNS4PGF_S_BCMnuVA@mail.gmail.com/3-jsonpath-fix-null-comparison.patch)
download | inline diff:
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 10ec66c6293..34420849eac 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -2128,14 +2128,15 @@ getJsonPathVariable(JsonPathExecContext *cxt, JsonPathItem *variable,
JsonbValue tmp;
JsonbValue *v;
- if (!vars)
- {
- value->type = jbvNull;
- return;
- }
-
Assert(variable->type == jpiVariable);
varName = jspGetString(variable, &varNameLength);
+
+ if (!vars)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find jsonpath variable \"%s\"",
+ pnstrdup(varName, varNameLength))));
+
tmp.type = jbvString;
tmp.val.string.val = varName;
tmp.val.string.len = varNameLength;
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-04-23 18:55 Andrey Borodin <[email protected]>
parent: Andrey Rachitskiy <[email protected]>
0 siblings, 1 reply; 13+ messages in thread
From: Andrey Borodin @ 2026-04-23 18:55 UTC (permalink / raw)
To: Andrey Rachitskiy <[email protected]>; +Cc: [email protected]; [email protected]
> On 20 Apr 2026, at 18:38, Andrey Rachitskiy <[email protected]> wrote:
>
> I propose a targeted backpatch for REL_14/15/16 in jsonpath_exec.c to align missing variable handling with newer branches and prevent pathological memory growth on malformed/hostile jsonpath expressions.
Hi! Thank you for the report and proposed fix. I've took a look into the
patch.
So we can use vars like this:
# SELECT jsonb_path_exists(
'{"x": 42}'::jsonb,
'$ ? ($"threshold" < 50)'::jsonpath,
'{"threshold": 10}'::jsonb -- HERE go vars
);
Operator @? is doing the same, but without supplied vars. And this thread
essentially points to buggy handling of vars:
# SELECT j @? '$"no_such_var"'
FROM (VALUES
('{"important": "data"}'::jsonb),
('42'::jsonb),
('null'::jsonb),
('false'::jsonb)
) AS t(j);
?column?
----------
t
t
t
t
(4 rows)
It basically says that path with value of var "no_such_var" exists everywhere.
I think it's a bug, but we would need a JSON Path expert here.
17+ throws an error, which seems suspicious to me too. @? is expected to
operate in silent mode. Perhaps, we should just return NULL instead of t.
By using RETURN_ERROR macro. But it might sound overly invasive for back
branches.
Even if we are going to throw an error, we can give mode details. I'd suggest
instead of "could not find jsonpath variable \"%s\"" throwing something like
"no variables supplied to reference by variable \"%s\"" or something along
those lines.
Besides this, the direction of the fix looks good to me. Thank you!
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-04-28 20:19 Nikita Malakhov <[email protected]>
parent: Andrey Borodin <[email protected]>
0 siblings, 1 reply; 13+ messages in thread
From: Nikita Malakhov @ 2026-04-28 20:19 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; Amit Langote <[email protected]>; +Cc: Andrey Rachitskiy <[email protected]>; [email protected]; [email protected]
Hi!
According to the Jsonpath standard, malformed expression should return an
error,
but not all cases of malformation are thoroughly described.
When this functionality was developed (Jsonpath and SQL/JSON) the absence
of the variable was considered as malformation and was decided to throw an
error
in threads long time ago. In case this behavior to be a subject for change
it surely
should not be backported, but the error-throwing code has to.
I agree that sometimes Json (-path) functionality provides very little info
on errors,
and it could be extended.
Changing this behavior is subject for Hackers and should be approved by Tom
Lane,
Adres Freund and other people responsible for including Jsonpath and
SQL/JSON
into Postgres.
On Tue, Apr 28, 2026 at 7:51 PM Andrey Borodin <[email protected]> wrote:
>
>
> > On 20 Apr 2026, at 18:38, Andrey Rachitskiy <[email protected]> wrote:
> >
> > I propose a targeted backpatch for REL_14/15/16 in jsonpath_exec.c to
> align missing variable handling with newer branches and prevent
> pathological memory growth on malformed/hostile jsonpath expressions.
>
> Hi! Thank you for the report and proposed fix. I've took a look into the
> patch.
>
> So we can use vars like this:
>
> # SELECT jsonb_path_exists(
> '{"x": 42}'::jsonb,
> '$ ? ($"threshold" < 50)'::jsonpath,
> '{"threshold": 10}'::jsonb -- HERE go vars
> );
>
> Operator @? is doing the same, but without supplied vars. And this thread
> essentially points to buggy handling of vars:
>
> # SELECT j @? '$"no_such_var"'
> FROM (VALUES
> ('{"important": "data"}'::jsonb),
> ('42'::jsonb),
> ('null'::jsonb),
> ('false'::jsonb)
> ) AS t(j);
> ?column?
> ----------
> t
> t
> t
> t
> (4 rows)
>
> It basically says that path with value of var "no_such_var" exists
> everywhere.
>
> I think it's a bug, but we would need a JSON Path expert here.
>
> 17+ throws an error, which seems suspicious to me too. @? is expected to
> operate in silent mode. Perhaps, we should just return NULL instead of t.
> By using RETURN_ERROR macro. But it might sound overly invasive for back
> branches.
>
> Even if we are going to throw an error, we can give mode details. I'd
> suggest
> instead of "could not find jsonpath variable \"%s\"" throwing something
> like
> "no variables supplied to reference by variable \"%s\"" or something along
> those lines.
>
>
> Besides this, the direction of the fix looks good to me. Thank you!
>
>
> Best regards, Andrey Borodin.
>
>
>
>
--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-04-30 12:03 Andrey Borodin <[email protected]>
parent: Nikita Malakhov <[email protected]>
0 siblings, 1 reply; 13+ messages in thread
From: Andrey Borodin @ 2026-04-30 12:03 UTC (permalink / raw)
To: Nikita Malakhov <[email protected]>; +Cc: Amit Langote <[email protected]>; Andrey Rachitskiy <[email protected]>; [email protected]; [email protected]
> On 29 Apr 2026, at 01:19, Nikita Malakhov <[email protected]> wrote:
>
> According to the Jsonpath standard, malformed expression should return an error,
> but not all cases of malformation are thoroughly described.
>
> When this functionality was developed (Jsonpath and SQL/JSON) the absence
> of the variable was considered as malformation and was decided to throw an error
> in threads long time ago. In case this behavior to be a subject for change it surely
> should not be backported, but the error-throwing code has to.
I think you just explained very well why we throw an error. Your arguments against
silent mode are valid and we don't need to consider RETURN_ERROR any further.
Current master behavior throws an error, in this thread author propose to backport it.
This might be behavior change for some users. But it seems to me we have to backport,
because
SELECT '42'::jsonb @? '$"no_such_var"';
should not return true. What do you think?
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-05-26 11:17 Nikita Malakhov <[email protected]>
parent: Andrey Borodin <[email protected]>
0 siblings, 1 reply; 13+ messages in thread
From: Nikita Malakhov @ 2026-05-26 11:17 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Amit Langote <[email protected]>; Andrey Rachitskiy <[email protected]>; [email protected]; [email protected]
Hi!
Sorry for the late answer - I missed this email.
No objections against backporting error throwing code, sorry if my
explanation
was not clear enough, of course OOM is very bad behavior and should be
corrected,
even if this would result in changes for some users (hopefully not).
My point in the previous message was only against silent mode.
Thanks!
On Thu, Apr 30, 2026 at 3:03 PM Andrey Borodin <[email protected]> wrote:
>
>
> > On 29 Apr 2026, at 01:19, Nikita Malakhov <[email protected]> wrote:
> >
> > According to the Jsonpath standard, malformed expression should return
> an error,
> > but not all cases of malformation are thoroughly described.
> >
> > When this functionality was developed (Jsonpath and SQL/JSON) the absence
> > of the variable was considered as malformation and was decided to throw
> an error
> > in threads long time ago. In case this behavior to be a subject for
> change it surely
> > should not be backported, but the error-throwing code has to.
>
> I think you just explained very well why we throw an error. Your arguments
> against
> silent mode are valid and we don't need to consider RETURN_ERROR any
> further.
>
> Current master behavior throws an error, in this thread author propose to
> backport it.
> This might be behavior change for some users. But it seems to me we have
> to backport,
> because
>
> SELECT '42'::jsonb @? '$"no_such_var"';
>
> should not return true. What do you think?
>
>
> Best regards, Andrey Borodin.
--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-05-31 13:33 Andrey Borodin <[email protected]>
parent: Nikita Malakhov <[email protected]>
0 siblings, 1 reply; 13+ messages in thread
From: Andrey Borodin @ 2026-05-31 13:33 UTC (permalink / raw)
To: Nikita Malakhov <[email protected]>; +Cc: Amit Langote <[email protected]>; Andrey Rachitskiy <[email protected]>; PostgreSQL mailing lists <[email protected]>; Nikolay Shaplov <[email protected]>
> On 26 May 2026, at 16:17, Nikita Malakhov <[email protected]> wrote:
>
> of course OOM is very bad behavior and should be corrected
I think the biggest problem is not OOM, but incorrect results for query
SELECT '42'::jsonb @? '$"no_such_var"';
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-06-05 08:58 Amit Langote <[email protected]>
parent: Andrey Borodin <[email protected]>
0 siblings, 1 reply; 13+ messages in thread
From: Amit Langote @ 2026-06-05 08:58 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Nikita Malakhov <[email protected]>; Andrey Rachitskiy <[email protected]>; PostgreSQL mailing lists <[email protected]>; Nikolay Shaplov <[email protected]>
Hi,
Before I dig into the patch properly after the weekend, one question
on the report itself: has anyone traced why the old path runs away on
memory? We've characterized it as missing-var, then null, then
evaluation continues, then OOM, but I don't think the actual growing
allocation has been pinned down. Mostly want to understand whether the
same runaway is reachable without a missing variable, since raising
the error early wouldn't catch those cases.
- Thanks, Amit
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-06-05 10:03 Andrey Rachitskiy <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 2 replies; 13+ messages in thread
From: Andrey Rachitskiy @ 2026-06-05 10:03 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Nikita Malakhov <[email protected]>; PostgreSQL mailing lists <[email protected]>; Nikolay Shaplov <[email protected]>
The growing allocation is leaked temporary JsonValueLists in
executePredicate() (local lseq/rseq, ~1482–1547) and the arithmetic helpers
executeBinaryArithmExpr() / executeUnaryArithmExpr() (~1561–1684). Each
nested comparison or arithmetic subexpression materializes operands via
executeItemOptUnwrapResult[NoThrow]() → executeNextItem() →
JsonValueListAppend() (~1165, ~2451), but the interim lists are never freed
before return. For @? specifically, executeJsonPath() also leaks a local
vals list in strict exists mode (~579–586).
Missing vars make the AFL case worse by returning null instead of error, so
evaluation continues deep into nested $?()/comparisons instead of stopping
at the first $"…" reference. The same leak mechanism is reachable without
missing vars — Tom Lane demonstrated this on master (5a2043bf713) with $[*]
? (@ < $) on a large array.
Our missing-variable patch fixes the reported OOM and the @? semantics bug
by aborting early. Whether REL_14/15/16 also need a broader fix for interim
JsonValueList cleanup is beyond what I can confidently propose; I've tried
to pin down where the growth happens for that discussion.
пт, 5 июн. 2026 г. в 13:58, Amit Langote <[email protected]>:
> Hi,
>
> Before I dig into the patch properly after the weekend, one question
> on the report itself: has anyone traced why the old path runs away on
> memory? We've characterized it as missing-var, then null, then
> evaluation continues, then OOM, but I don't think the actual growing
> allocation has been pinned down. Mostly want to understand whether the
> same runaway is reachable without a missing variable, since raising
> the error early wouldn't catch those cases.
>
> - Thanks, Amit
>
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-06-05 10:57 Nikita Malakhov <[email protected]>
parent: Andrey Rachitskiy <[email protected]>
1 sibling, 0 replies; 13+ messages in thread
From: Nikita Malakhov @ 2026-06-05 10:57 UTC (permalink / raw)
To: Andrey Rachitskiy <[email protected]>; +Cc: Amit Langote <[email protected]>; Andrey Borodin <[email protected]>; PostgreSQL mailing lists <[email protected]>; Nikolay Shaplov <[email protected]>
Hi!
Thank you very much for this investigation! I'd take a look into the patch
after the weekend.
On Fri, Jun 5, 2026 at 1:03 PM Andrey Rachitskiy <[email protected]> wrote:
> The growing allocation is leaked temporary JsonValueLists in
> executePredicate() (local lseq/rseq, ~1482–1547) and the arithmetic helpers
> executeBinaryArithmExpr() / executeUnaryArithmExpr() (~1561–1684). Each
> nested comparison or arithmetic subexpression materializes operands via
> executeItemOptUnwrapResult[NoThrow]() → executeNextItem() →
> JsonValueListAppend() (~1165, ~2451), but the interim lists are never freed
> before return. For @? specifically, executeJsonPath() also leaks a local
> vals list in strict exists mode (~579–586).
>
> Missing vars make the AFL case worse by returning null instead of error,
> so evaluation continues deep into nested $?()/comparisons instead of
> stopping at the first $"…" reference. The same leak mechanism is reachable
> without missing vars — Tom Lane demonstrated this on master (5a2043bf713)
> with $[*] ? (@ < $) on a large array.
>
> Our missing-variable patch fixes the reported OOM and the @? semantics bug
> by aborting early. Whether REL_14/15/16 also need a broader fix for interim
> JsonValueList cleanup is beyond what I can confidently propose; I've tried
> to pin down where the growth happens for that discussion.
>
> пт, 5 июн. 2026 г. в 13:58, Amit Langote <[email protected]>:
>
>> Hi,
>>
>> Before I dig into the patch properly after the weekend, one question
>> on the report itself: has anyone traced why the old path runs away on
>> memory? We've characterized it as missing-var, then null, then
>> evaluation continues, then OOM, but I don't think the actual growing
>> allocation has been pinned down. Mostly want to understand whether the
>> same runaway is reachable without a missing variable, since raising
>> the error early wouldn't catch those cases.
>>
>> - Thanks, Amit
>>
>
--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-06-17 08:27 Amit Langote <[email protected]>
parent: Andrey Rachitskiy <[email protected]>
1 sibling, 1 reply; 13+ messages in thread
From: Amit Langote @ 2026-06-17 08:27 UTC (permalink / raw)
To: Andrey Rachitskiy <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Nikita Malakhov <[email protected]>; PostgreSQL mailing lists <[email protected]>; Nikolay Shaplov <[email protected]>
Hi Andrey,
On Fri, Jun 5, 2026 at 7:03 PM Andrey Rachitskiy <[email protected]> wrote:
> The growing allocation is leaked temporary JsonValueLists in executePredicate() (local lseq/rseq, ~1482–1547) and the arithmetic helpers executeBinaryArithmExpr() / executeUnaryArithmExpr() (~1561–1684). Each nested comparison or arithmetic subexpression materializes operands via executeItemOptUnwrapResult[NoThrow]() → executeNextItem() → JsonValueListAppend() (~1165, ~2451), but the interim lists are never freed before return. For @? specifically, executeJsonPath() also leaks a local vals list in strict exists mode (~579–586).
>
> Missing vars make the AFL case worse by returning null instead of error, so evaluation continues deep into nested $?()/comparisons instead of stopping at the first $"…" reference. The same leak mechanism is reachable without missing vars — Tom Lane demonstrated this on master (5a2043bf713) with $[*] ? (@ < $) on a large array.
>
> Our missing-variable patch fixes the reported OOM and the @? semantics bug by aborting early. Whether REL_14/15/16 also need a broader fix for interim JsonValueList cleanup is beyond what I can confidently propose; I've tried to pin down where the growth happens for that discussion.
Thanks for that tracedown and for pointing to Tom's commit. The deeper
interim-JsonValueList leak looks unlikely to get fixed in the back
branches; Tom's cleanup (5a2043bf713) went only to master.
I'll look at committing the attached revised version of your Apr 20
patch (same fix, plus a regression test) down to REL_14. Please
check/test.
--
Thanks, Amit Langote
Attachments:
[application/octet-stream] v2-0001-Report-undefined-jsonpath-variable-when-no-variab.patch (4.7K, ../../CA+HiwqEL=Hr3ReVLqRy-U7JFOmx8ziS=_NLQcD4sA411kS5K9A@mail.gmail.com/2-v2-0001-Report-undefined-jsonpath-variable-when-no-variab.patch)
download | inline diff:
From d52b3b5a728d5130e3e14acd778c5b608b9e50f3 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 17 Jun 2026 17:21:38 +0900
Subject: [PATCH v2] Report undefined jsonpath variable when no variables are
supplied
The two-argument jsonb @? and @@ operators invoke the jsonpath executor
with no variable set. In that case getJsonPathVariable() treated any
"$name" reference as JSON null and continued evaluating, instead of
reporting the variable as undefined.
This produced incorrect results -- for example '42'::jsonb @? '$"x"'
returned true -- and, for some malformed or hostile jsonpath expressions
with deeply nested predicates, allowed essentially unbounded memory
consumption that could get the backend killed by the OOM killer.
Report the undefined variable as an error in this case as well, reusing
the message already emitted when a variable is not found among supplied
variables. This matches the behavior of v17 and later, where the
jsonpath executor was reorganized. Stopping at the first undefined
variable reference also resolves the reported memory-growth case.
Note this is a user-visible change in the back branches: a jsonpath
expression that references a variable while no variables are supplied now
raises an error rather than silently evaluating it as NULL. The previous
behavior was incorrect, so the change is judged worthwhile.
Author: Andrey Rachitskiy <[email protected]>
Reported-by: Andrey Rachitskiy <[email protected]>
Reviewed-by: Andrey Borodin <[email protected]>
Reviewed-by: Nikita Malakhov <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14
---
src/backend/utils/adt/jsonpath_exec.c | 13 +++++++------
src/test/regress/expected/jsonb_jsonpath.out | 7 +++++++
src/test/regress/sql/jsonb_jsonpath.sql | 5 +++++
3 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 10ec66c6293..34420849eac 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -2128,14 +2128,15 @@ getJsonPathVariable(JsonPathExecContext *cxt, JsonPathItem *variable,
JsonbValue tmp;
JsonbValue *v;
- if (!vars)
- {
- value->type = jbvNull;
- return;
- }
-
Assert(variable->type == jpiVariable);
varName = jspGetString(variable, &varNameLength);
+
+ if (!vars)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find jsonpath variable \"%s\"",
+ pnstrdup(varName, varNameLength))));
+
tmp.type = jbvString;
tmp.val.string.val = varName;
tmp.val.string.len = varNameLength;
diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out
index 6659bc9091a..2ab90643450 100644
--- a/src/test/regress/expected/jsonb_jsonpath.out
+++ b/src/test/regress/expected/jsonb_jsonpath.out
@@ -487,6 +487,13 @@ select * from jsonb_path_query('{"a": 10}', '$');
select * from jsonb_path_query('{"a": 10}', '$ ? (@.a < $value)');
ERROR: could not find jsonpath variable "value"
+-- bug #19458: the @? and @@ operators supply no variables, so a variable
+-- reference must be reported as undefined rather than silently treated as
+-- NULL (the latter gave wrong results and could drive unbounded memory use)
+select jsonb '42' @? '$"no_such_var"';
+ERROR: could not find jsonpath variable "no_such_var"
+select jsonb '42' @@ '$"no_such_var" == 1';
+ERROR: could not find jsonpath variable "no_such_var"
select * from jsonb_path_query('{"a": 10}', '$ ? (@.a < $value)', '1');
ERROR: "vars" argument is not an object
DETAIL: Jsonpath parameters should be encoded as key-value pairs of "vars" object.
diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql
index e0ce509264a..f0e08d27e84 100644
--- a/src/test/regress/sql/jsonb_jsonpath.sql
+++ b/src/test/regress/sql/jsonb_jsonpath.sql
@@ -98,6 +98,11 @@ select jsonb_path_query('[1,2,3]', '$[last ? (@.type() == "string")]', silent =>
select * from jsonb_path_query('{"a": 10}', '$');
select * from jsonb_path_query('{"a": 10}', '$ ? (@.a < $value)');
+-- bug #19458: the @? and @@ operators supply no variables, so a variable
+-- reference must be reported as undefined rather than silently treated as
+-- NULL (the latter gave wrong results and could drive unbounded memory use)
+select jsonb '42' @? '$"no_such_var"';
+select jsonb '42' @@ '$"no_such_var" == 1';
select * from jsonb_path_query('{"a": 10}', '$ ? (@.a < $value)', '1');
select * from jsonb_path_query('{"a": 10}', '$ ? (@.a < $value)', '[{"value" : 13}]');
select * from jsonb_path_query('{"a": 10}', '$ ? (@.a < $value)', '{"value" : 13}');
--
2.47.3
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-06-17 09:57 Andrey Rachitskiy <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 13+ messages in thread
From: Andrey Rachitskiy @ 2026-06-17 09:57 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Nikita Malakhov <[email protected]>; PostgreSQL mailing lists <[email protected]>; Nikolay Shaplov <[email protected]>
Hi Amit,
I tested patch
v2-0001-Report-undefined-jsonpath-variable-when-no-variab.patch on:
REL_14_STABLE
REL_15_STABLE
REL_16_STABLE
Results:
The patch applies cleanly on all three branches.
Regression tests added/affected by the patch pass on all three branches:
REL_14: all relevant tests passed
REL_15: all relevant tests passed
REL_16: all relevant tests passed
The provided jsonpath reproducer query now consistently returns an error
about an undefined jsonpath variable on all three branches, which matches
the expected behavior change from this patch.
Conclusion:
The patch looks good from the testing side and can be merged.
ср, 17 июн. 2026 г. в 13:27, Amit Langote <[email protected]>:
> Hi Andrey,
>
> On Fri, Jun 5, 2026 at 7:03 PM Andrey Rachitskiy <[email protected]>
> wrote:
> > The growing allocation is leaked temporary JsonValueLists in
> executePredicate() (local lseq/rseq, ~1482–1547) and the arithmetic helpers
> executeBinaryArithmExpr() / executeUnaryArithmExpr() (~1561–1684). Each
> nested comparison or arithmetic subexpression materializes operands via
> executeItemOptUnwrapResult[NoThrow]() → executeNextItem() →
> JsonValueListAppend() (~1165, ~2451), but the interim lists are never freed
> before return. For @? specifically, executeJsonPath() also leaks a local
> vals list in strict exists mode (~579–586).
> >
> > Missing vars make the AFL case worse by returning null instead of error,
> so evaluation continues deep into nested $?()/comparisons instead of
> stopping at the first $"…" reference. The same leak mechanism is reachable
> without missing vars — Tom Lane demonstrated this on master (5a2043bf713)
> with $[*] ? (@ < $) on a large array.
> >
> > Our missing-variable patch fixes the reported OOM and the @? semantics
> bug by aborting early. Whether REL_14/15/16 also need a broader fix for
> interim JsonValueList cleanup is beyond what I can confidently propose;
> I've tried to pin down where the growth happens for that discussion.
>
> Thanks for that tracedown and for pointing to Tom's commit. The deeper
> interim-JsonValueList leak looks unlikely to get fixed in the back
> branches; Tom's cleanup (5a2043bf713) went only to master.
>
> I'll look at committing the attached revised version of your Apr 20
> patch (same fix, plus a regression test) down to REL_14. Please
> check/test.
>
> --
> Thanks, Amit Langote
>
--
---
Regards,
Andrey Rachitskiy
Postgres Professional
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
@ 2026-06-18 06:57 Amit Langote <[email protected]>
parent: Andrey Rachitskiy <[email protected]>
0 siblings, 0 replies; 13+ messages in thread
From: Amit Langote @ 2026-06-18 06:57 UTC (permalink / raw)
To: Andrey Rachitskiy <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Nikita Malakhov <[email protected]>; PostgreSQL mailing lists <[email protected]>; Nikolay Shaplov <[email protected]>
On Wed, Jun 17, 2026 at 6:58 PM Andrey Rachitskiy <[email protected]> wrote:
>
> Hi Amit,
>
> I tested patch v2-0001-Report-undefined-jsonpath-variable-when-no-variab.patch on:
>
> REL_14_STABLE
> REL_15_STABLE
> REL_16_STABLE
>
> Results:
> The patch applies cleanly on all three branches.
> Regression tests added/affected by the patch pass on all three branches:
> REL_14: all relevant tests passed
> REL_15: all relevant tests passed
> REL_16: all relevant tests passed
> The provided jsonpath reproducer query now consistently returns an error about an undefined jsonpath variable on all three branches, which matches the expected behavior change from this patch.
>
> Conclusion:
> The patch looks good from the testing side and can be merged.
Thanks for checking, pushed.
--
Thanks, Amit Langote
^ permalink raw reply [nested|flat] 13+ messages in thread
end of thread, other threads:[~2026-06-18 06:57 UTC | newest]
Thread overview: 13+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-04-17 11:20 BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables PG Bug reporting form <[email protected]>
2026-04-20 13:38 ` Andrey Rachitskiy <[email protected]>
2026-04-23 18:55 ` Andrey Borodin <[email protected]>
2026-04-28 20:19 ` Nikita Malakhov <[email protected]>
2026-04-30 12:03 ` Andrey Borodin <[email protected]>
2026-05-26 11:17 ` Nikita Malakhov <[email protected]>
2026-05-31 13:33 ` Andrey Borodin <[email protected]>
2026-06-05 08:58 ` Amit Langote <[email protected]>
2026-06-05 10:03 ` Andrey Rachitskiy <[email protected]>
2026-06-05 10:57 ` Nikita Malakhov <[email protected]>
2026-06-17 08:27 ` Amit Langote <[email protected]>
2026-06-17 09:57 ` Andrey Rachitskiy <[email protected]>
2026-06-18 06:57 ` Amit Langote <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox