public inbox for [email protected]
help / color / mirror / Atom feedBUG #19525: In `contrib/dict_int`, handling a token whose first byte is a null byte causes `pnstrdup()` .
4+ messages / 4 participants
[nested] [flat]
* BUG #19525: In `contrib/dict_int`, handling a token whose first byte is a null byte causes `pnstrdup()` .
@ 2026-06-18 07:54 PG Bug reporting form <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: PG Bug reporting form @ 2026-06-18 07:54 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]
The following bug has been logged on the website:
Bug reference: 19525
Logged by: Yuelin Wang
Email address: [email protected]
PostgreSQL version: 19beta1
Operating system: Linux (Ubuntu 24.04, x86_64)
Description:
**Component**: `contrib/dict_int/dict_int.c`, function `dintdict_lexize()`
(line 109)
Requires a `SQL_ASCII`-encoded database (to bypass null-byte encoding
checks) and superuser to install the extension and create a helper function
that passes a `bytea` token directly to the lexize callback. Once the
dictionary is created, any role granted `EXECUTE` on the helper can trigger
the crash.
```sql
-- 1. Create SQL_ASCII database (null bytes are not rejected)
CREATE DATABASE vuln_ascii ENCODING 'SQL_ASCII' TEMPLATE template0;
\c vuln_ascii
-- 2. Install extension and create an intdict dictionary with
REJECTLONG=false
CREATE EXTENSION dict_int;
CREATE TEXT SEARCH DICTIONARY intdict_test (
TEMPLATE = intdict_template,
MAXLEN = 8192,
REJECTLONG = false
);
-- 3. Create a C helper (raw_lexize.so) that invokes the lexize callback
with
-- a raw bytea token, bypassing the text encoding layer.
CREATE FUNCTION raw_lexize(dict regdictionary, token bytea)
RETURNS text[] AS 'raw_lexize', 'raw_lexize' LANGUAGE C STRICT;
-- 4. Trigger: null byte at position 0 causes pnstrdup to allocate 1 byte,
-- but txt[8192] = '\0' writes 8191 bytes past the end of the allocation.
SELECT raw_lexize('intdict_test',
decode('00' || repeat('78', 10000), 'hex'));
-- Server closes connection; ASan reports heap-buffer-overflow WRITE of size
1
-- at dict_int.c:109 in dintdict_lexize.
```
ASan confirmation (server killed the backend; connection dropped):
```
==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x525000052880
WRITE of size 1 at 0x525000052880 thread T0
#0 in dintdict_lexize
/data/ylwang/Projects/postgres/contrib/dict_int/dict_int.c:109
#1 in FunctionCall4Coll .../src/backend/utils/fmgr/fmgr.c:1215
#2 in raw_lexize /tmp/raw_lexize.c:37
SUMMARY: AddressSanitizer: heap-buffer-overflow
.../contrib/dict_int/dict_int.c:109 in dintdict_lexize
```
`pnstrdup(ptr, len)` uses `strnlen(ptr, len)` internally, so when the token
begins with a null byte it allocates only 1 byte. The variable `len` is not
updated to reflect this and retains the original token length, so the guard
at line 98 (`if (len > d->maxlen)`) passes, and line 109 writes `'\0'` at
offset `d->maxlen` (e.g., 8192) into a 1-byte allocation.
The fix is to recompute the effective length from the allocated buffer after
the `pnstrdup` call, for example by replacing the `if (len > d->maxlen)`
check with `if (strlen(txt) > d->maxlen)`. This ensures the truncation
offset is always within the bounds of what `pnstrdup` actually allocated.
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: BUG #19525: In `contrib/dict_int`, handling a token whose first byte is a null byte causes `pnstrdup()` .
@ 2026-06-18 14:41 Ayush Tiwari <[email protected]>
parent: PG Bug reporting form <[email protected]>
0 siblings, 2 replies; 4+ messages in thread
From: Ayush Tiwari @ 2026-06-18 14:41 UTC (permalink / raw)
To: [email protected]; [email protected]
Hi,
On Thu, 18 Jun 2026 at 18:54, PG Bug reporting form <[email protected]>
wrote:
> The following bug has been logged on the website:
>
> Bug reference: 19525
> Logged by: Yuelin Wang
> Email address: [email protected]
> PostgreSQL version: 19beta1
> Operating system: Linux (Ubuntu 24.04, x86_64)
> Description:
>
> **Component**: `contrib/dict_int/dict_int.c`, function `dintdict_lexize()`
> (line 109)
>
> Requires a `SQL_ASCII`-encoded database (to bypass null-byte encoding
> checks) and superuser to install the extension and create a helper function
> that passes a `bytea` token directly to the lexize callback. Once the
> dictionary is created, any role granted `EXECUTE` on the helper can trigger
> the crash.
>
> ```sql
> -- 1. Create SQL_ASCII database (null bytes are not rejected)
> CREATE DATABASE vuln_ascii ENCODING 'SQL_ASCII' TEMPLATE template0;
> \c vuln_ascii
>
> -- 2. Install extension and create an intdict dictionary with
> REJECTLONG=false
> CREATE EXTENSION dict_int;
> CREATE TEXT SEARCH DICTIONARY intdict_test (
> TEMPLATE = intdict_template,
> MAXLEN = 8192,
> REJECTLONG = false
> );
>
> -- 3. Create a C helper (raw_lexize.so) that invokes the lexize callback
> with
> -- a raw bytea token, bypassing the text encoding layer.
> CREATE FUNCTION raw_lexize(dict regdictionary, token bytea)
> RETURNS text[] AS 'raw_lexize', 'raw_lexize' LANGUAGE C STRICT;
>
> -- 4. Trigger: null byte at position 0 causes pnstrdup to allocate 1 byte,
> -- but txt[8192] = '\0' writes 8191 bytes past the end of the
> allocation.
> SELECT raw_lexize('intdict_test',
> decode('00' || repeat('78', 10000), 'hex'));
> -- Server closes connection; ASan reports heap-buffer-overflow WRITE of
> size
> 1
> -- at dict_int.c:109 in dintdict_lexize.
> ```
>
> ASan confirmation (server killed the backend; connection dropped):
> ```
> ==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x525000052880
> WRITE of size 1 at 0x525000052880 thread T0
> #0 in dintdict_lexize
> /data/ylwang/Projects/postgres/contrib/dict_int/dict_int.c:109
> #1 in FunctionCall4Coll .../src/backend/utils/fmgr/fmgr.c:1215
> #2 in raw_lexize /tmp/raw_lexize.c:37
> SUMMARY: AddressSanitizer: heap-buffer-overflow
> .../contrib/dict_int/dict_int.c:109 in dintdict_lexize
>
Thanks for the report and repro!
`pnstrdup(ptr, len)` uses `strnlen(ptr, len)` internally, so when the token
> begins with a null byte it allocates only 1 byte. The variable `len` is not
> updated to reflect this and retains the original token length, so the guard
> at line 98 (`if (len > d->maxlen)`) passes, and line 109 writes `'\0'` at
> offset `d->maxlen` (e.g., 8192) into a 1-byte allocation.
>
> The fix is to recompute the effective length from the allocated buffer
> after
> the `pnstrdup` call, for example by replacing the `if (len > d->maxlen)`
> check with `if (strlen(txt) > d->maxlen)`. This ensures the truncation
> offset is always within the bounds of what `pnstrdup` actually allocated.
>
Your analysis seems right to me.
While looking around I think dict_xsyn may have a related issue: in
dxsyn_lexize() the token is copied with pnstrdup() and the original
length is then handed to str_tolower(), which reads that many bytes and
so could read past the shorter copy.
Attaching a patch that fixes both the above issues.
Regards,
Ayush
Attachments:
[application/x-patch] 0001-Fix-out-of-bounds-access-on-embedded-null-tokens-in-.patch (2.5K, ../../CAJTYsWW7+aVAFFV3dxg1s-RrtwioYPj1qQ9o2oAM40MReTXzAg@mail.gmail.com/3-0001-Fix-out-of-bounds-access-on-embedded-null-tokens-in-.patch)
download | inline diff:
From 382c736b599292470b050e112b66945c46a5a7e7 Mon Sep 17 00:00:00 2001
From: Ayush Tiwari <[email protected]>
Date: Thu, 18 Jun 2026 19:54:54 +0530
Subject: [PATCH] Fix out-of-bounds access on embedded null tokens in dict_int
and dict_xsyn
pnstrdup() stops at the first null byte, so a token beginning with a null
is copied as a 1-byte string. Both dictionaries then reused the original
token length against that shorter copy.
dintdict_lexize() (dict_int) used it for the "len > maxlen" check and then
wrote txt[maxlen] = '\0' past the end of the 1-byte allocation; recompute
len from the copy. dxsyn_lexize() (dict_xsyn) passed the original length
to str_tolower(), which then read past the copy; pass the token directly
instead, dropping the needless copy.
Reaching either path needs a token with a null byte, which the normal
text-search input path does not produce (it requires SQL_ASCII and a
caller passing raw bytes to the lexize method).
Reported-by: Yuelin Wang <[email protected]>
Bug: #19525
---
contrib/dict_int/dict_int.c | 6 ++++++
contrib/dict_xsyn/dict_xsyn.c | 15 +++++++--------
2 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/contrib/dict_int/dict_int.c b/contrib/dict_int/dict_int.c
index aafce894977..b4413cfde99 100644
--- a/contrib/dict_int/dict_int.c
+++ b/contrib/dict_int/dict_int.c
@@ -95,6 +95,12 @@ dintdict_lexize(PG_FUNCTION_ARGS)
else
txt = pnstrdup(in, len);
+ /*
+ * pnstrdup() stops at the first null byte, so recompute len from the copy;
+ * otherwise the truncation below could write past the allocation.
+ */
+ len = strlen(txt);
+
if (len > d->maxlen)
{
if (d->rejectlong)
diff --git a/contrib/dict_xsyn/dict_xsyn.c b/contrib/dict_xsyn/dict_xsyn.c
index 9e3784e0f47..b375e8f7387 100644
--- a/contrib/dict_xsyn/dict_xsyn.c
+++ b/contrib/dict_xsyn/dict_xsyn.c
@@ -211,14 +211,13 @@ dxsyn_lexize(PG_FUNCTION_ARGS)
if (!length || d->len == 0)
PG_RETURN_POINTER(NULL);
- /* Create search pattern */
- {
- char *temp = pnstrdup(in, length);
-
- word.key = str_tolower(temp, length, DEFAULT_COLLATION_OID);
- pfree(temp);
- word.value = NULL;
- }
+ /*
+ * Pass the token directly to str_tolower(); copying it first with
+ * pnstrdup() and reusing the original length would read past the copy if
+ * the token contains a null byte.
+ */
+ word.key = str_tolower(in, length, DEFAULT_COLLATION_OID);
+ word.value = NULL;
/* Look for matching syn */
found = (Syn *) bsearch(&word, d->syn, d->len, sizeof(Syn), compare_syn);
--
2.34.1
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: BUG #19525: In `contrib/dict_int`, handling a token whose first byte is a null byte causes `pnstrdup()` .
@ 2026-06-18 18:19 王跃林 <[email protected]>
parent: Ayush Tiwari <[email protected]>
1 sibling, 0 replies; 4+ messages in thread
From: 王跃林 @ 2026-06-18 18:19 UTC (permalink / raw)
To: Ayush Tiwari <[email protected]>; +Cc: pgsql-bugs <[email protected]>
The fix looks correct. Recomputing len from the copy via strlen(txt) after pnstrdup() in dict_int directly addresses the root cause I reported. The dict_xsyn fix is also a clean approach since skipping the intermediate copy avoids the length mismatch entirely. Thank you for the patch!
王跃林
[email protected]
Original:
From：Ayush Tiwari <[email protected]>Date：2026-06-18 22:41:32(中国 (GMT+08:00))To：3020001251<[email protected]> , pgsql-bugs<[email protected]>Cc：Subject：Re: BUG #19525: In `contrib/dict_int`, handling a token whose first byte is a null byte causes `pnstrdup()` .Hi,
On Thu, 18 Jun 2026 at 18:54, PG Bug reporting form <[email protected]> wrote:
The following bug has been logged on the website:
Bug reference: 19525
Logged by: Yuelin Wang
Email address: [email protected]
PostgreSQL version: 19beta1
Operating system: Linux (Ubuntu 24.04, x86_64)
Description:
**Component**: `contrib/dict_int/dict_int.c`, function `dintdict_lexize()`
(line 109)
Requires a `SQL_ASCII`-encoded database (to bypass null-byte encoding
checks) and superuser to install the extension and create a helper function
that passes a `bytea` token directly to the lexize callback. Once the
dictionary is created, any role granted `EXECUTE` on the helper can trigger
the crash.
```sql
-- 1. Create SQL_ASCII database (null bytes are not rejected)
CREATE DATABASE vuln_ascii ENCODING 'SQL_ASCII' TEMPLATE template0;
\c vuln_ascii
-- 2. Install extension and create an intdict dictionary with
REJECTLONG=false
CREATE EXTENSION dict_int;
CREATE TEXT SEARCH DICTIONARY intdict_test (
TEMPLATE = intdict_template,
MAXLEN = 8192,
REJECTLONG = false
);
-- 3. Create a C helper (raw_lexize.so) that invokes the lexize callback
with
-- a raw bytea token, bypassing the text encoding layer.
CREATE FUNCTION raw_lexize(dict regdictionary, token bytea)
RETURNS text[] AS 'raw_lexize', 'raw_lexize' LANGUAGE C STRICT;
-- 4. Trigger: null byte at position 0 causes pnstrdup to allocate 1 byte,
-- but txt[8192] = '\0' writes 8191 bytes past the end of the allocation.
SELECT raw_lexize('intdict_test',
decode('00' || repeat('78', 10000), 'hex'));
-- Server closes connection; ASan reports heap-buffer-overflow WRITE of size
1
-- at dict_int.c:109 in dintdict_lexize.
```
ASan confirmation (server killed the backend; connection dropped):
```
==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x525000052880
WRITE of size 1 at 0x525000052880 thread T0
#0 in dintdict_lexize
/data/ylwang/Projects/postgres/contrib/dict_int/dict_int.c:109
#1 in FunctionCall4Coll .../src/backend/utils/fmgr/fmgr.c:1215
#2 in raw_lexize /tmp/raw_lexize.c:37
SUMMARY: AddressSanitizer: heap-buffer-overflow
.../contrib/dict_int/dict_int.c:109 in dintdict_lexize
Thanks for the report and repro!
`pnstrdup(ptr, len)` uses `strnlen(ptr, len)` internally, so when the token
begins with a null byte it allocates only 1 byte. The variable `len` is not
updated to reflect this and retains the original token length, so the guard
at line 98 (`if (len > d->maxlen)`) passes, and line 109 writes `'\0'` at
offset `d->maxlen` (e.g., 8192) into a 1-byte allocation.
The fix is to recompute the effective length from the allocated buffer after
the `pnstrdup` call, for example by replacing the `if (len > d->maxlen)`
check with `if (strlen(txt) > d->maxlen)`. This ensures the truncation
offset is always within the bounds of what `pnstrdup` actually allocated.
Your analysis seems right to me.
While looking around I think dict_xsyn may have a related issue: in
dxsyn_lexize() the token is copied with pnstrdup() and the original
length is then handed to str_tolower(), which reads that many bytes and
so could read past the shorter copy.
Attaching a patch that fixes both the above issues.
Regards,
Ayush
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: BUG #19525: In `contrib/dict_int`, handling a token whose first byte is a null byte causes `pnstrdup()` .
@ 2026-07-05 20:12 Tom Lane <[email protected]>
parent: Ayush Tiwari <[email protected]>
1 sibling, 0 replies; 4+ messages in thread
From: Tom Lane @ 2026-07-05 20:12 UTC (permalink / raw)
To: Ayush Tiwari <[email protected]>; +Cc: [email protected]; [email protected]
Ayush Tiwari <[email protected]> writes:
> On Thu, 18 Jun 2026 at 18:54, PG Bug reporting form <[email protected]>
> wrote:
>> -- 3. Create a C helper (raw_lexize.so) that invokes the lexize callback
>> with
>> -- a raw bytea token, bypassing the text encoding layer.
>> CREATE FUNCTION raw_lexize(dict regdictionary, token bytea)
>> RETURNS text[] AS 'raw_lexize', 'raw_lexize' LANGUAGE C STRICT;
> Your analysis seems right to me.
It is not. I do not think this is a bug; if it is a bug we have got
probably hundreds of places that'd have to be fixed. The general rule
in Postgres is that we do not permit embedded nulls in text values.
The fact that a misguided superuser can break that rule doesn't make
it a bug for code to depend on it. (There are approximately an
infinite number of ways for a misguided superuser to break Postgres,
just as the root user can destroy a Unix system many times over.)
So I see no value in this proposed change in dintdict_lexize.
But the one in dxsyn_lexize seems worthwhile, just because
it makes that code shorter, simpler, and faster. Since that's
mostly an example module, I don't feel a need to back-patch,
but I'll go commit that to HEAD.
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-07-05 20:12 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-06-18 07:54 BUG #19525: In `contrib/dict_int`, handling a token whose first byte is a null byte causes `pnstrdup()` . PG Bug reporting form <[email protected]>
2026-06-18 14:41 ` Ayush Tiwari <[email protected]>
2026-06-18 18:19 ` 王跃林 <[email protected]>
2026-07-05 20:12 ` Tom Lane <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox