MIME-Version: 1.0
References: <19525-b0be8e4eb7dbaf07@postgresql.org>
In-Reply-To: <19525-b0be8e4eb7dbaf07@postgresql.org>
From: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Date: Thu, 18 Jun 2026 20:11:32 +0530
Message-ID: 
 <CAJTYsWW7+aVAFFV3dxg1s-RrtwioYPj1qQ9o2oAM40MReTXzAg@mail.gmail.com>
Subject: Re: BUG #19525: In `contrib/dict_int`, handling a token whose first
 byte is a null byte causes `pnstrdup()` .
To: 3020001251@tju.edu.cn, pgsql-bugs@lists.postgresql.org
Content-Type: multipart/mixed; boundary="000000000000c82d150654882cc1"
Archived-At: 
 <https://www.postgresql.org/message-id/CAJTYsWW7%2BaVAFFV3dxg1s-RrtwioYPj1qQ9o2oAM40MReTXzAg%40mail.gmail.com>
Precedence: bulk

--000000000000c82d150654882cc1
Content-Type: multipart/alternative; boundary="000000000000c82d130654882cbf"

--000000000000c82d130654882cbf
Content-Type: text/plain; charset="UTF-8"

Hi,

On Thu, 18 Jun 2026 at 18:54, PG Bug reporting form <noreply@postgresql.org>
wrote:

> The following bug has been logged on the website:
>
> Bug reference:      19525
> Logged by:          Yuelin Wang
> Email address:      3020001251@tju.edu.cn
> PostgreSQL version: 19beta1
> Operating system:   Linux (Ubuntu 24.04, x86_64)
> Description:
>
> **Component**: `contrib/dict_int/dict_int.c`, function `dintdict_lexize()`
> (line 109)
>
> Requires a `SQL_ASCII`-encoded database (to bypass null-byte encoding
> checks) and superuser to install the extension and create a helper function
> that passes a `bytea` token directly to the lexize callback. Once the
> dictionary is created, any role granted `EXECUTE` on the helper can trigger
> the crash.
>
> ```sql
> -- 1. Create SQL_ASCII database (null bytes are not rejected)
> CREATE DATABASE vuln_ascii ENCODING 'SQL_ASCII' TEMPLATE template0;
> \c vuln_ascii
>
> -- 2. Install extension and create an intdict dictionary with
> REJECTLONG=false
> CREATE EXTENSION dict_int;
> CREATE TEXT SEARCH DICTIONARY intdict_test (
>     TEMPLATE = intdict_template,
>     MAXLEN = 8192,
>     REJECTLONG = false
> );
>
> -- 3. Create a C helper (raw_lexize.so) that invokes the lexize callback
> with
> --    a raw bytea token, bypassing the text encoding layer.
> CREATE FUNCTION raw_lexize(dict regdictionary, token bytea)
>     RETURNS text[] AS 'raw_lexize', 'raw_lexize' LANGUAGE C STRICT;
>
> -- 4. Trigger: null byte at position 0 causes pnstrdup to allocate 1 byte,
> --    but txt[8192] = '\0' writes 8191 bytes past the end of the
> allocation.
> SELECT raw_lexize('intdict_test',
>     decode('00' || repeat('78', 10000), 'hex'));
> -- Server closes connection; ASan reports heap-buffer-overflow WRITE of
> size
> 1
> -- at dict_int.c:109 in dintdict_lexize.
> ```
>
> ASan confirmation (server killed the backend; connection dropped):
> ```
> ==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x525000052880
> WRITE of size 1 at 0x525000052880 thread T0
>     #0 in dintdict_lexize
> /data/ylwang/Projects/postgres/contrib/dict_int/dict_int.c:109
>     #1 in FunctionCall4Coll .../src/backend/utils/fmgr/fmgr.c:1215
>     #2 in raw_lexize /tmp/raw_lexize.c:37
> SUMMARY: AddressSanitizer: heap-buffer-overflow
> .../contrib/dict_int/dict_int.c:109 in dintdict_lexize
>

Thanks for the report and repro!

`pnstrdup(ptr, len)` uses `strnlen(ptr, len)` internally, so when the token
> begins with a null byte it allocates only 1 byte. The variable `len` is not
> updated to reflect this and retains the original token length, so the guard
> at line 98 (`if (len > d->maxlen)`) passes, and line 109 writes `'\0'` at
> offset `d->maxlen` (e.g., 8192) into a 1-byte allocation.
>
> The fix is to recompute the effective length from the allocated buffer
> after
> the `pnstrdup` call, for example by replacing the `if (len > d->maxlen)`
> check with `if (strlen(txt) > d->maxlen)`. This ensures the truncation
> offset is always within the bounds of what `pnstrdup` actually allocated.
>

Your analysis seems right to me.

While looking around I think dict_xsyn may have a related issue: in
dxsyn_lexize() the token is copied with pnstrdup() and the original
length is then handed to str_tolower(), which reads that many bytes and
so could read past the shorter copy.

Attaching a patch that fixes both the above issues.

Regards,
Ayush

--000000000000c82d130654882cbf
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr">Hi,</div><br><div class=
=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, 18 Jun 2026 =
at 18:54, PG Bug reporting form &lt;<a href=3D"mailto:noreply@postgresql.or=
g" target=3D"_blank">noreply@postgresql.org</a>&gt; wrote:<br></div><blockq=
uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1p=
x solid rgb(204,204,204);padding-left:1ex">The following bug has been logge=
d on the website:<br>
<br>
Bug reference:=C2=A0 =C2=A0 =C2=A0 19525<br>
Logged by:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Yuelin Wang<br>
Email address:=C2=A0 =C2=A0 =C2=A0 <a href=3D"mailto:3020001251@tju.edu.cn"=
 target=3D"_blank">3020001251@tju.edu.cn</a><br>
PostgreSQL version: 19beta1<br>
Operating system:=C2=A0 =C2=A0Linux (Ubuntu 24.04, x86_64)<br>
Description:=C2=A0 =C2=A0 =C2=A0 =C2=A0 <br>
<br>
**Component**: `contrib/dict_int/dict_int.c`, function `dintdict_lexize()`<=
br>
(line 109)<br>
<br>
Requires a `SQL_ASCII`-encoded database (to bypass null-byte encoding<br>
checks) and superuser to install the extension and create a helper function=
<br>
that passes a `bytea` token directly to the lexize callback. Once the<br>
dictionary is created, any role granted `EXECUTE` on the helper can trigger=
<br>
the crash.<br>
<br>
```sql<br>
-- 1. Create SQL_ASCII database (null bytes are not rejected)<br>
CREATE DATABASE vuln_ascii ENCODING &#39;SQL_ASCII&#39; TEMPLATE template0;=
<br>
\c vuln_ascii<br>
<br>
-- 2. Install extension and create an intdict dictionary with<br>
REJECTLONG=3Dfalse<br>
CREATE EXTENSION dict_int;<br>
CREATE TEXT SEARCH DICTIONARY intdict_test (<br>
=C2=A0 =C2=A0 TEMPLATE =3D intdict_template,<br>
=C2=A0 =C2=A0 MAXLEN =3D 8192,<br>
=C2=A0 =C2=A0 REJECTLONG =3D false<br>
);<br>
<br>
-- 3. Create a C helper (raw_lexize.so) that invokes the lexize callback<br=
>
with<br>
--=C2=A0 =C2=A0 a raw bytea token, bypassing the text encoding layer.<br>
CREATE FUNCTION raw_lexize(dict regdictionary, token bytea)<br>
=C2=A0 =C2=A0 RETURNS text[] AS &#39;raw_lexize&#39;, &#39;raw_lexize&#39; =
LANGUAGE C STRICT;<br>
<br>
-- 4. Trigger: null byte at position 0 causes pnstrdup to allocate 1 byte,<=
br>
--=C2=A0 =C2=A0 but txt[8192] =3D &#39;\0&#39; writes 8191 bytes past the e=
nd of the allocation.<br>
SELECT raw_lexize(&#39;intdict_test&#39;,<br>
=C2=A0 =C2=A0 decode(&#39;00&#39; || repeat(&#39;78&#39;, 10000), &#39;hex&=
#39;));<br>
-- Server closes connection; ASan reports heap-buffer-overflow WRITE of siz=
e<br>
1<br>
-- at dict_int.c:109 in dintdict_lexize.<br>
```<br>
<br>
ASan confirmation (server killed the backend; connection dropped):<br>
```<br>
=3D=3DERROR: AddressSanitizer: heap-buffer-overflow on address 0x5250000528=
80<br>
WRITE of size 1 at 0x525000052880 thread T0<br>
=C2=A0 =C2=A0 #0 in dintdict_lexize<br>
/data/ylwang/Projects/postgres/contrib/dict_int/dict_int.c:109<br>
=C2=A0 =C2=A0 #1 in FunctionCall4Coll .../src/backend/utils/fmgr/fmgr.c:121=
5<br>
=C2=A0 =C2=A0 #2 in raw_lexize /tmp/raw_lexize.c:37<br>
SUMMARY: AddressSanitizer: heap-buffer-overflow<br>
.../contrib/dict_int/dict_int.c:109 in dintdict_lexize<br></blockquote><div=
><br>Thanks for the report and repro!=C2=A0<br><br></div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg=
b(204,204,204);padding-left:1ex">
`pnstrdup(ptr, len)` uses `strnlen(ptr, len)` internally, so when the token=
<br>
begins with a null byte it allocates only 1 byte. The variable `len` is not=
<br>
updated to reflect this and retains the original token length, so the guard=
<br>
at line 98 (`if (len &gt; d-&gt;maxlen)`) passes, and line 109 writes `&#39=
;\0&#39;` at<br>
offset `d-&gt;maxlen` (e.g., 8192) into a 1-byte allocation.<br>
<br>
The fix is to recompute the effective length from the allocated buffer afte=
r<br>
the `pnstrdup` call, for example by replacing the `if (len &gt; d-&gt;maxle=
n)`<br>
check with `if (strlen(txt) &gt; d-&gt;maxlen)`. This ensures the truncatio=
n<br>
offset is always within the bounds of what `pnstrdup` actually allocated.<b=
r></blockquote><div><br>Your analysis seems right to me.<br><br>While looki=
ng around I think dict_xsyn may have a related issue: in<br>dxsyn_lexize() =
the token is copied with pnstrdup() and the original<br>length is then hand=
ed to str_tolower(), which reads that many bytes and<br>so could read past =
the shorter copy.<br><br>Attaching a patch that fixes both the above issues=
.<br><br>Regards,<br>Ayush=C2=A0</div></div></div>
</div>

--000000000000c82d130654882cbf--

--000000000000c82d150654882cc1
Content-Type: application/x-patch;
	name="0001-Fix-out-of-bounds-access-on-embedded-null-tokens-in-.patch"
Content-Disposition: attachment;
	filename="0001-Fix-out-of-bounds-access-on-embedded-null-tokens-in-.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mqjlqjh90>
X-Attachment-Id: f_mqjlqjh90

RnJvbSAzODJjNzM2YjU5OTI5MjQ3MGIwNTBlMTEyYjY2OTQ1YzQ2YTVhN2U3IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBBeXVzaCBUaXdhcmkgPGF5dXNodGl3YXJpLnNsZzAxQGdtYWls
LmNvbT4KRGF0ZTogVGh1LCAxOCBKdW4gMjAyNiAxOTo1NDo1NCArMDUzMApTdWJqZWN0OiBbUEFU
Q0hdIEZpeCBvdXQtb2YtYm91bmRzIGFjY2VzcyBvbiBlbWJlZGRlZCBudWxsIHRva2VucyBpbiBk
aWN0X2ludAogYW5kIGRpY3RfeHN5bgoKcG5zdHJkdXAoKSBzdG9wcyBhdCB0aGUgZmlyc3QgbnVs
bCBieXRlLCBzbyBhIHRva2VuIGJlZ2lubmluZyB3aXRoIGEgbnVsbAppcyBjb3BpZWQgYXMgYSAx
LWJ5dGUgc3RyaW5nLiAgQm90aCBkaWN0aW9uYXJpZXMgdGhlbiByZXVzZWQgdGhlIG9yaWdpbmFs
CnRva2VuIGxlbmd0aCBhZ2FpbnN0IHRoYXQgc2hvcnRlciBjb3B5LgoKZGludGRpY3RfbGV4aXpl
KCkgKGRpY3RfaW50KSB1c2VkIGl0IGZvciB0aGUgImxlbiA+IG1heGxlbiIgY2hlY2sgYW5kIHRo
ZW4Kd3JvdGUgdHh0W21heGxlbl0gPSAnXDAnIHBhc3QgdGhlIGVuZCBvZiB0aGUgMS1ieXRlIGFs
bG9jYXRpb247IHJlY29tcHV0ZQpsZW4gZnJvbSB0aGUgY29weS4gIGR4c3luX2xleGl6ZSgpIChk
aWN0X3hzeW4pIHBhc3NlZCB0aGUgb3JpZ2luYWwgbGVuZ3RoCnRvIHN0cl90b2xvd2VyKCksIHdo
aWNoIHRoZW4gcmVhZCBwYXN0IHRoZSBjb3B5OyBwYXNzIHRoZSB0b2tlbiBkaXJlY3RseQppbnN0
ZWFkLCBkcm9wcGluZyB0aGUgbmVlZGxlc3MgY29weS4KClJlYWNoaW5nIGVpdGhlciBwYXRoIG5l
ZWRzIGEgdG9rZW4gd2l0aCBhIG51bGwgYnl0ZSwgd2hpY2ggdGhlIG5vcm1hbAp0ZXh0LXNlYXJj
aCBpbnB1dCBwYXRoIGRvZXMgbm90IHByb2R1Y2UgKGl0IHJlcXVpcmVzIFNRTF9BU0NJSSBhbmQg
YQpjYWxsZXIgcGFzc2luZyByYXcgYnl0ZXMgdG8gdGhlIGxleGl6ZSBtZXRob2QpLgoKUmVwb3J0
ZWQtYnk6IFl1ZWxpbiBXYW5nIDwzMDIwMDAxMjUxQHRqdS5lZHUuY24+CkJ1ZzogIzE5NTI1Ci0t
LQogY29udHJpYi9kaWN0X2ludC9kaWN0X2ludC5jICAgfCAgNiArKysrKysKIGNvbnRyaWIvZGlj
dF94c3luL2RpY3RfeHN5bi5jIHwgMTUgKysrKysrKy0tLS0tLS0tCiAyIGZpbGVzIGNoYW5nZWQs
IDEzIGluc2VydGlvbnMoKyksIDggZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvY29udHJpYi9k
aWN0X2ludC9kaWN0X2ludC5jIGIvY29udHJpYi9kaWN0X2ludC9kaWN0X2ludC5jCmluZGV4IGFh
ZmNlODk0OTc3Li5iNDQxM2NmZGU5OSAxMDA2NDQKLS0tIGEvY29udHJpYi9kaWN0X2ludC9kaWN0
X2ludC5jCisrKyBiL2NvbnRyaWIvZGljdF9pbnQvZGljdF9pbnQuYwpAQCAtOTUsNiArOTUsMTIg
QEAgZGludGRpY3RfbGV4aXplKFBHX0ZVTkNUSU9OX0FSR1MpCiAJZWxzZQogCQl0eHQgPSBwbnN0
cmR1cChpbiwgbGVuKTsKIAorCS8qCisJICogcG5zdHJkdXAoKSBzdG9wcyBhdCB0aGUgZmlyc3Qg
bnVsbCBieXRlLCBzbyByZWNvbXB1dGUgbGVuIGZyb20gdGhlIGNvcHk7CisJICogb3RoZXJ3aXNl
IHRoZSB0cnVuY2F0aW9uIGJlbG93IGNvdWxkIHdyaXRlIHBhc3QgdGhlIGFsbG9jYXRpb24uCisJ
ICovCisJbGVuID0gc3RybGVuKHR4dCk7CisKIAlpZiAobGVuID4gZC0+bWF4bGVuKQogCXsKIAkJ
aWYgKGQtPnJlamVjdGxvbmcpCmRpZmYgLS1naXQgYS9jb250cmliL2RpY3RfeHN5bi9kaWN0X3hz
eW4uYyBiL2NvbnRyaWIvZGljdF94c3luL2RpY3RfeHN5bi5jCmluZGV4IDllMzc4NGUwZjQ3Li5i
Mzc1ZThmNzM4NyAxMDA2NDQKLS0tIGEvY29udHJpYi9kaWN0X3hzeW4vZGljdF94c3luLmMKKysr
IGIvY29udHJpYi9kaWN0X3hzeW4vZGljdF94c3luLmMKQEAgLTIxMSwxNCArMjExLDEzIEBAIGR4
c3luX2xleGl6ZShQR19GVU5DVElPTl9BUkdTKQogCWlmICghbGVuZ3RoIHx8IGQtPmxlbiA9PSAw
KQogCQlQR19SRVRVUk5fUE9JTlRFUihOVUxMKTsKIAotCS8qIENyZWF0ZSBzZWFyY2ggcGF0dGVy
biAqLwotCXsKLQkJY2hhcgkgICAqdGVtcCA9IHBuc3RyZHVwKGluLCBsZW5ndGgpOwotCi0JCXdv
cmQua2V5ID0gc3RyX3RvbG93ZXIodGVtcCwgbGVuZ3RoLCBERUZBVUxUX0NPTExBVElPTl9PSUQp
OwotCQlwZnJlZSh0ZW1wKTsKLQkJd29yZC52YWx1ZSA9IE5VTEw7Ci0JfQorCS8qCisJICogUGFz
cyB0aGUgdG9rZW4gZGlyZWN0bHkgdG8gc3RyX3RvbG93ZXIoKTsgY29weWluZyBpdCBmaXJzdCB3
aXRoCisJICogcG5zdHJkdXAoKSBhbmQgcmV1c2luZyB0aGUgb3JpZ2luYWwgbGVuZ3RoIHdvdWxk
IHJlYWQgcGFzdCB0aGUgY29weSBpZgorCSAqIHRoZSB0b2tlbiBjb250YWlucyBhIG51bGwgYnl0
ZS4KKwkgKi8KKwl3b3JkLmtleSA9IHN0cl90b2xvd2VyKGluLCBsZW5ndGgsIERFRkFVTFRfQ09M
TEFUSU9OX09JRCk7CisJd29yZC52YWx1ZSA9IE5VTEw7CiAKIAkvKiBMb29rIGZvciBtYXRjaGlu
ZyBzeW4gKi8KIAlmb3VuZCA9IChTeW4gKikgYnNlYXJjaCgmd29yZCwgZC0+c3luLCBkLT5sZW4s
IHNpemVvZihTeW4pLCBjb21wYXJlX3N5bik7Ci0tIAoyLjM0LjEKCg==
--000000000000c82d150654882cc1--