public inbox for [email protected]
help / color / mirror / Atom feedBUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals
3+ messages / 3 participants
[nested] [flat]
* BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals
@ 2026-06-19 04:40 PG Bug reporting form <[email protected]>
2026-06-20 15:18 ` Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals Fujii Masao <[email protected]>
0 siblings, 1 reply; 3+ messages in thread
From: PG Bug reporting form @ 2026-06-19 04:40 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]
The following bug has been logged on the website:
Bug reference: 19528
Logged by: Yuelin Wang
Email address: [email protected]
PostgreSQL version: 19beta1
Operating system: Linux (Ubuntu 24.04, x86_64)
Description:
The buffer is allocated at line 2841 with a per-location budget of 10 bytes:
```c
norm_query_buflen = query_len + jstate->clocations_count * 10;
```
The comment explains the budget: a `$n` placeholder is at most 11 bytes and
the original constant is at least 1 byte, so net expansion is at most 10
bytes per location. This holds for non-squashed constants.
For squashed array elements, the sprintf at line 2883 writes `"$N /*, ...
*/"` instead of just `"$N"`:
```c
n_quer_loc += sprintf(norm_query + n_quer_loc, "$%d%s",
num_constants_replaced + 1 +
jstate->highest_extern_param_id,
locs[i].squashed ? " /*, ... */" : "");
```
`" /*, ... */"` is 11 bytes, so a squashed entry writes `"$N"` (2 bytes)
plus `" /*, ... */"` (11 bytes) = 13 bytes to replace a 1-byte constant, a
net expansion of 12 bytes. The budget of 10 bytes is exceeded by 2 bytes per
squashed location. With N squashed array elements the buffer overflows by 2N
bytes.
Any `ARRAY[a,b]` literal records its second element as a squashed
`clocations` entry. Ten such arrays produce a 20-byte overflow, which is
sufficient to trigger the Assert at line 2908:
```
TRAP: failed Assert("n_quer_loc <= norm_query_buflen"),
File: "pg_stat_statements.c", Line: 2908
```
In a Release build (assertions disabled) the overflow is a true heap write
past the `palloc` allocation, corrupting adjacent allocator metadata.
### Reproduction
`pg_stat_statements` must be listed in `shared_preload_libraries`. The
triggering role holds no special privileges beyond `LOGIN`.
```sql
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
CREATE ROLE vuln_004_lowpriv LOGIN PASSWORD 'vuln004';
GRANT pg_read_all_stats TO vuln_004_lowpriv;
SET ROLE vuln_004_lowpriv;
SELECT ARRAY[1,2], ARRAY[1,2], ARRAY[1,2], ARRAY[1,2], ARRAY[1,2],
ARRAY[1,2], ARRAY[1,2], ARRAY[1,2], ARRAY[1,2], ARRAY[1,2];
```
### Observed Output
psql output:
```
SET
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
connection to server was lost
```
Server log (PostgreSQL 19beta1, cassert build, 2026-06-19):
```
TRAP: failed Assert("n_quer_loc <= norm_query_buflen"), File:
"pg_stat_statements.c", Line: 2908, PID: 2671635
postgres: yuelinwang postgres [local]
SELECT(ExceptionalCondition+0x103)[0x5602a92f55b8]
/data/.../pg_stat_statements.so(+0x655b)[0x7d147957e55b]
/data/.../pg_stat_statements.so(+0x9ef7)[0x7d1479581ef7]
/data/.../pg_stat_statements.so(+0xb3ba)[0x7d14795833ba]
postgres: yuelinwang postgres [local]
SELECT(parse_analyze_fixedparams+0x120)[...]
postgres: yuelinwang postgres [local] SELECT(PostgresMain+0x1142)[...]
LOG: client backend (PID 2671635) was terminated by signal 6: Aborted
DETAIL: Failed process was running: SELECT ARRAY[1,2], ARRAY[1,2],
ARRAY[1,2], ARRAY[1,2], ARRAY[1,2],
ARRAY[1,2], ARRAY[1,2], ARRAY[1,2], ARRAY[1,2], ARRAY[1,2];
LOG: terminating any other active server processes
LOG: all server processes terminated; reinitializing
LOG: database system was interrupted; last known up at 2026-06-19 12:34:57
+08
LOG: database system was not properly shut down; automatic recovery in
progress
LOG: database system is ready to accept connections
```
### Expected vs Actual
| Step | Expected | Actual |
|---|---|---|
| SELECT with 10 ARRAY[1,2] literals | returns one row of 10 arrays |
backend crashes (SIGABRT signal 6) |
| Server log | nothing | `TRAP: failed Assert("n_quer_loc <=
norm_query_buflen"), File: "pg_stat_statements.c", Line: 2908` |
| Server state | unaffected | crash recovery triggered, postmaster
reinitializes |
| Connection | stays open | `server closed the connection unexpectedly` |
### Fix
Account for the extra 11 bytes appended when a squashed suffix is written.
One correct approach:
```c
norm_query_buflen = query_len + jstate->clocations_count *
(10 + (jstate->has_squashed_lists ? 11 : 0));
```
A simpler conservative fix that covers the worst case (`"$2147483648 /*, ...
*/"` = 22 bytes replacing a 1-byte constant, net 21 bytes):
```c
norm_query_buflen = query_len + jstate->clocations_count * 22;
```
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals
2026-06-19 04:40 BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals PG Bug reporting form <[email protected]>
@ 2026-06-20 15:18 ` Fujii Masao <[email protected]>
2026-06-20 16:30 ` Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals 王跃林 <[email protected]>
0 siblings, 1 reply; 3+ messages in thread
From: Fujii Masao @ 2026-06-20 15:18 UTC (permalink / raw)
To: [email protected]; [email protected]
On Sat, Jun 20, 2026 at 7:53 PM PG Bug reporting form
<[email protected]> wrote:
> `" /*, ... */"` is 11 bytes, so a squashed entry writes `"$N"` (2 bytes)
> plus `" /*, ... */"` (11 bytes) = 13 bytes to replace a 1-byte constant, a
> net expansion of 12 bytes. The budget of 10 bytes is exceeded by 2 bytes per
> squashed location. With N squashed array elements the buffer overflows by 2N
> bytes.
This analysis doesn't seem correct to me.
For "ARRAY[1,2]", the recorded location covers the entire list text,
"1,2" (3 bytes), which is replaced with "$n /*, ... */". For a
single-digit placeholder, "$1 /*, ... */" is 13 bytes, so replacing
"1,2" results in a net growth of 10 bytes.
That still fits within the existing budget. The budget is first
exceeded when the placeholder number reaches two digits:
"$10 /*, ... */" is 14 bytes, giving a net growth of 11 bytes.
Therefore, the smallest reproducer is 10 squashed "ARRAY[1,2]"
entries, where the shortfall seems only 1 byte overall, not 2 bytes per
squashed location.
> ### Fix
>
> Account for the extra 11 bytes appended when a squashed suffix is written.
> One correct approach:
>
> ```c
> norm_query_buflen = query_len + jstate->clocations_count *
> (10 + (jstate->has_squashed_lists ? 11 : 0));
> ```
>
> A simpler conservative fix that covers the worst case (`"$2147483648 /*, ...
> */"` = 22 bytes replacing a 1-byte constant, net 21 bytes):
>
> ```c
> norm_query_buflen = query_len + jstate->clocations_count * 22;
> ```
How about keeping the current "clocations_count * 10" budget for
ordinary replacements, counting squashed entries separately, and
adding only the extra space they may need, for example:
clocations_count * 10 + squashed_count * 9
The existing "+10" already covers ordinary placeholders. For squashed
entries, the worst-case additional overhead is 9 bytes, from replacing
the shortest possible source text "1,2" (3 bytes) with the longest
possible placeholder "$2147483647 /*, ... */" (22 bytes), for a total
growth of 19 bytes. Attached patch implements this approach.
Thoughts?
Regards,
--
Fujii Masao
Attachments:
[application/octet-stream] v1-0001-pg_stat_statements-add-room-for-squashed-placehol.patch (5.1K, ../../CAHGQGwEGpf3eWyLMqQcxP11NyhvBNCFzmZP1EXM5F1Ca4NaLMQ@mail.gmail.com/2-v1-0001-pg_stat_statements-add-room-for-squashed-placehol.patch)
download | inline diff:
From 0b22d7130d7f1315aa3d916520f2dd4aa0cfa91b Mon Sep 17 00:00:00 2001
From: Fujii Masao <[email protected]>
Date: Sat, 20 Jun 2026 23:11:41 +0900
Subject: [PATCH v1] pg_stat_statements: add room for squashed placeholders
---
.../pg_stat_statements/expected/squashing.out | 24 +++++++++++++++++++
.../pg_stat_statements/pg_stat_statements.c | 17 ++++++++++++-
contrib/pg_stat_statements/sql/squashing.sql | 7 ++++++
3 files changed, 47 insertions(+), 1 deletion(-)
diff --git a/contrib/pg_stat_statements/expected/squashing.out b/contrib/pg_stat_statements/expected/squashing.out
index 8438235a2ce..057f56c4741 100644
--- a/contrib/pg_stat_statements/expected/squashing.out
+++ b/contrib/pg_stat_statements/expected/squashing.out
@@ -79,6 +79,30 @@ SELECT query, calls FROM pg_stat_statements ORDER BY query COLLATE "C";
SELECT pg_stat_statements_reset() IS NOT NULL AS t | 1
(3 rows)
+-- small array literals still fit once squashed placeholders reach 2 digits
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+SELECT ARRAY[1,2] AS a1, ARRAY[1,2] AS a2, ARRAY[1,2] AS a3, ARRAY[1,2] AS a4,
+ ARRAY[1,2] AS a5, ARRAY[1,2] AS a6, ARRAY[1,2] AS a7, ARRAY[1,2] AS a8,
+ ARRAY[1,2] AS a9, ARRAY[1,2] AS a10;
+ a1 | a2 | a3 | a4 | a5 | a6 | a7 | a8 | a9 | a10
+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------
+ {1,2} | {1,2} | {1,2} | {1,2} | {1,2} | {1,2} | {1,2} | {1,2} | {1,2} | {1,2}
+(1 row)
+
+SELECT query, calls FROM pg_stat_statements ORDER BY query COLLATE "C";
+ query | calls
+------------------------------------------------------------------------------------------------------------------------+-------
+ SELECT ARRAY[$1 /*, ... */] AS a1, ARRAY[$2 /*, ... */] AS a2, ARRAY[$3 /*, ... */] AS a3, ARRAY[$4 /*, ... */] AS a4,+| 1
+ ARRAY[$5 /*, ... */] AS a5, ARRAY[$6 /*, ... */] AS a6, ARRAY[$7 /*, ... */] AS a7, ARRAY[$8 /*, ... */] AS a8,+|
+ ARRAY[$9 /*, ... */] AS a9, ARRAY[$10 /*, ... */] AS a10 |
+ SELECT pg_stat_statements_reset() IS NOT NULL AS t | 1
+(2 rows)
+
-- built-in functions will be squashed
-- the IN and ARRAY forms of this statement will have the same queryId
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 92315627916..bd79a7d1470 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -2822,6 +2822,7 @@ generate_normalized_query(const JumbleState *jstate, const char *query,
last_off = 0, /* Offset from start for previous tok */
last_tok_len = 0; /* Length (in bytes) of that tok */
int num_constants_replaced = 0;
+ int squashed_count = 0;
LocationLen *locs = NULL;
/*
@@ -2837,8 +2838,22 @@ generate_normalized_query(const JumbleState *jstate, const char *query,
* certainly isn't more than 11 bytes, even if n reaches INT_MAX. We
* could refine that limit based on the max value of n for the current
* query, but it hardly seems worth any extra effort to do so.
+ *
+ * Squashed lists need a little more room. The shortest squashed source
+ * text is "1,2" (three bytes), while the longest squashed placeholder is
+ * 22 bytes long (an INT_MAX placeholder plus the squash marker), so each
+ * squashed list can need up to 9 bytes more than the base allowance of 10
+ * bytes per entry.
*/
- norm_query_buflen = query_len + jstate->clocations_count * 10;
+ for (int i = 0; i < jstate->clocations_count; i++)
+ {
+ /* Ignore duplicate locations that ComputeConstantLengths() disabled */
+ if (locs[i].squashed && locs[i].length >= 0)
+ squashed_count++;
+ }
+
+ norm_query_buflen = query_len + jstate->clocations_count * 10 +
+ squashed_count * 9;
/* Allocate result buffer */
norm_query = palloc(norm_query_buflen + 1);
diff --git a/contrib/pg_stat_statements/sql/squashing.sql b/contrib/pg_stat_statements/sql/squashing.sql
index fc9e6573873..2bd26b24eec 100644
--- a/contrib/pg_stat_statements/sql/squashing.sql
+++ b/contrib/pg_stat_statements/sql/squashing.sql
@@ -24,6 +24,13 @@ SELECT ARRAY[1, 2, 3, 4];
SELECT ARRAY[1, 2, 3, 4, 5];
SELECT query, calls FROM pg_stat_statements ORDER BY query COLLATE "C";
+-- small array literals still fit once squashed placeholders reach 2 digits
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+SELECT ARRAY[1,2] AS a1, ARRAY[1,2] AS a2, ARRAY[1,2] AS a3, ARRAY[1,2] AS a4,
+ ARRAY[1,2] AS a5, ARRAY[1,2] AS a6, ARRAY[1,2] AS a7, ARRAY[1,2] AS a8,
+ ARRAY[1,2] AS a9, ARRAY[1,2] AS a10;
+SELECT query, calls FROM pg_stat_statements ORDER BY query COLLATE "C";
+
-- built-in functions will be squashed
-- the IN and ARRAY forms of this statement will have the same queryId
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
--
2.53.0
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals
2026-06-19 04:40 BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals PG Bug reporting form <[email protected]>
2026-06-20 15:18 ` Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals Fujii Masao <[email protected]>
@ 2026-06-20 16:30 ` 王跃林 <[email protected]>
0 siblings, 0 replies; 3+ messages in thread
From: 王跃林 @ 2026-06-20 16:30 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: pgsql-bugs <[email protected]>
The analysis looks correct to me; clocations_count * 10 + squashed_count * 9 tightly covers the worst case and is cleaner than the alternatives.
王跃林
[email protected]
Original:
From：Fujii Masao <[email protected]>Date：2026-06-20 23:18:19(中国 (GMT+08:00))To：3020001251<[email protected]> , pgsql-bugs<[email protected]>Cc：Subject：Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array LiteralsOn Sat, Jun 20, 2026 at 7:53 PM PG Bug reporting form
<[email protected]> wrote:
> `" /*, ... */"` is 11 bytes, so a squashed entry writes `"$N"` (2 bytes)
> plus `" /*, ... */"` (11 bytes) = 13 bytes to replace a 1-byte constant, a
> net expansion of 12 bytes. The budget of 10 bytes is exceeded by 2 bytes per
> squashed location. With N squashed array elements the buffer overflows by 2N
> bytes.
This analysis doesn't seem correct to me.
For "ARRAY[1,2]", the recorded location covers the entire list text,
"1,2" (3 bytes), which is replaced with "$n /*, ... */". For a
single-digit placeholder, "$1 /*, ... */" is 13 bytes, so replacing
"1,2" results in a net growth of 10 bytes.
That still fits within the existing budget. The budget is first
exceeded when the placeholder number reaches two digits:
"$10 /*, ... */" is 14 bytes, giving a net growth of 11 bytes.
Therefore, the smallest reproducer is 10 squashed "ARRAY[1,2]"
entries, where the shortfall seems only 1 byte overall, not 2 bytes per
squashed location.
> ### Fix
>
> Account for the extra 11 bytes appended when a squashed suffix is written.
> One correct approach:
>
> ```c
> norm_query_buflen = query_len + jstate->clocations_count *
> (10 + (jstate->has_squashed_lists ? 11 : 0));
> ```
>
> A simpler conservative fix that covers the worst case (`"$2147483648 /*, ...
> */"` = 22 bytes replacing a 1-byte constant, net 21 bytes):
>
> ```c
> norm_query_buflen = query_len + jstate->clocations_count * 22;
> ```
How about keeping the current "clocations_count * 10" budget for
ordinary replacements, counting squashed entries separately, and
adding only the extra space they may need, for example:
clocations_count * 10 + squashed_count * 9
The existing "+10" already covers ordinary placeholders. For squashed
entries, the worst-case additional overhead is 9 bytes, from replacing
the shortest possible source text "1,2" (3 bytes) with the longest
possible placeholder "$2147483647 /*, ... */" (22 bytes), for a total
growth of 19 bytes. Attached patch implements this approach.
Thoughts?
Regards,
--
Fujii Masao
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2026-06-20 16:30 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-06-19 04:40 BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals PG Bug reporting form <[email protected]>
2026-06-20 15:18 ` Fujii Masao <[email protected]>
2026-06-20 16:30 ` 王跃林 <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox