pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedResidual tsquery cleanups
3+ messages / 2 participants
[nested] [flat]
* Residual tsquery cleanups
@ 2026-08-16 16:21 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 3+ messages in thread
From: Tom Lane @ 2026-08-16 16:21 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org
The two attached, pretty trivial patches clean up some loose ends
from the recent tsvector/tsquery security commits. I'm thinking
of back-patching both, but perhaps there is an argument for not
back-patching the first one?
regards, tom lane
Attachments:
[text/x-diff] v1-0001-Tighten-up-tsqueryrecv.patch (1.8K, ../../455079.1786897319@sss.pgh.pa.us/2-v1-0001-Tighten-up-tsqueryrecv.patch)
download | inline diff:
From 5567ea2fb4ba82a5ebe4efaacef58715d95c9216 Mon Sep 17 00:00:00 2001
From: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun, 16 Aug 2026 12:08:22 -0400
Subject: [PATCH v1 1/2] Tighten up tsqueryrecv().
tsqueryrecv() accepted zero-length lexemes, which tsqueryin() doesn't.
It also accepted phrase distance values larger than MAXENTRYPOS,
which tsqueryin() doesn't. While neither of these omissions are
very harmful in themselves, they do allow accepting tsquery values
that will fail in a subsequent textual dump/reload.
Commit 23d9ad771 performed similar tightening of tsvectorrecv(),
but I left off these changes at the time because they didn't seem
to have security implications.
Reported-by: Claude Code (via Noah Misch)
Author: Tom Lane <tgl@sss.pgh.pa.us>
Backpatch-through: 14
---
src/backend/utils/adt/tsquery.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/tsquery.c b/src/backend/utils/adt/tsquery.c
index 3247d5db082..df26fc2a139 100644
--- a/src/backend/utils/adt/tsquery.c
+++ b/src/backend/utils/adt/tsquery.c
@@ -1272,6 +1272,9 @@ tsqueryrecv(PG_FUNCTION_ARGS)
if (weight > 0xF)
elog(ERROR, "invalid tsquery: invalid weight bitmap");
+ if (val_len == 0)
+ elog(ERROR, "invalid tsquery: empty operand");
+
if (val_len > MAXSTRLEN)
elog(ERROR, "invalid tsquery: operand too long");
@@ -1311,7 +1314,14 @@ tsqueryrecv(PG_FUNCTION_ARGS)
item->qoperator.oper = oper;
if (oper == OP_PHRASE)
- item->qoperator.distance = (int16) pq_getmsgint(buf, sizeof(int16));
+ {
+ unsigned int dist = pq_getmsgint(buf, sizeof(int16));
+
+ if (dist > MAXENTRYPOS)
+ elog(ERROR, "invalid tsquery: invalid phrase distance %d",
+ dist);
+ item->qoperator.distance = (int16) dist;
+ }
}
else
elog(ERROR, "unrecognized tsquery node type: %d", item->type);
--
2.52.0
[text/x-diff] v1-0002-Clean-up-documentation-about-text-search-datatype.patch (2.5K, ../../455079.1786897319@sss.pgh.pa.us/3-v1-0002-Clean-up-documentation-about-text-search-datatype.patch)
download | inline diff:
From 40adfc381b50c29014d05ee497c424a8ee7ac533 Mon Sep 17 00:00:00 2001
From: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun, 16 Aug 2026 12:15:26 -0400
Subject: [PATCH v1 2/2] Clean up documentation about text search datatype
limits.
textsearch.sgml neglected to mention that the MAXSTRPOS total-length
limit applies to tsquery as well as tsvector. It also claims that
there is a 32K limit on the total number of nodes in a tsquery, which
is wrong. (I suspect that QueryOperator.left may once have been
int16, which would give rise to such a limit. But it's uint32 now,
so you'd hit the 1GB varlena limit well before overflowing that.)
While at it, re-order the bullet points into an order that makes
more sense, to me anyway.
Reported-by: Claude Code (via Noah Misch)
Author: Tom Lane <tgl@sss.pgh.pa.us>
Backpatch-through: 14
---
doc/src/sgml/textsearch.sgml | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/textsearch.sgml b/doc/src/sgml/textsearch.sgml
index d6d2ddeaacc..2f3a76bef1e 100644
--- a/doc/src/sgml/textsearch.sgml
+++ b/doc/src/sgml/textsearch.sgml
@@ -4046,24 +4046,24 @@ Parser: "pg_catalog.default"
<para>The length of each lexeme must be less than 2 kilobytes</para>
</listitem>
<listitem>
- <para>The length of a <type>tsvector</type> (lexemes + positions) must be
- less than 1 megabyte</para>
+ <para>No more than 256 positions per lexeme</para>
</listitem>
<listitem>
<para>Position values in <type>tsvector</type> must be greater than 0 and
no more than 16,383</para>
</listitem>
<listitem>
- <para>The match distance in a <literal><<replaceable>N</replaceable>></literal>
- (FOLLOWED BY) <type>tsquery</type> operator cannot be more than
- 16,384</para>
+ <para>The length of a <type>tsvector</type>'s data (lexemes + positions)
+ must be less than 1 megabyte</para>
</listitem>
<listitem>
- <para>No more than 256 positions per lexeme</para>
+ <para>The length of a <type>tsquery</type>'s data (lexemes only)
+ must be less than 1 megabyte</para>
</listitem>
<listitem>
- <para>The number of nodes (lexemes + operators) in a <type>tsquery</type>
- must be less than 32,768</para>
+ <para>The match distance in a <literal><<replaceable>N</replaceable>></literal>
+ (FOLLOWED BY) <type>tsquery</type> operator cannot be more than
+ 16,384</para>
</listitem>
</itemizedlist>
</para>
--
2.52.0
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Residual tsquery cleanups
@ 2026-08-17 02:39 Chao Li <li.evan.chao@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 3+ messages in thread
From: Chao Li @ 2026-08-17 02:39 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@lists.postgresql.org
> On Aug 17, 2026, at 00:21, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> The two attached, pretty trivial patches clean up some loose ends
> from the recent tsvector/tsquery security commits. I'm thinking
> of back-patching both, but perhaps there is an argument for not
> back-patching the first one?
>
> regards, tom lane
>
Overall looks good to me. A few small comments:
1 - 0001
```
+ unsigned int dist = pq_getmsgint(buf, sizeof(int16));
+
+ if (dist > MAXENTRYPOS)
+ elog(ERROR, "invalid tsquery: invalid phrase distance %d",
+ dist);
```
Here, “dist” is an unsigned int, so maybe it would be better to use “%u” than “%d” to print it.
2 - 0002
···
+ <para>The length of a <type>tsquery</type>'s data (lexemes only)
+ must be less than 1 megabyte</para>
···
Look like to me this is inaccurate, because in pushValue_internal(), the "if (distance > MAXSTRPOS)” check happens before adding a new element, thus distance may exceed MAXSTRPOS after the element is added.
The doc also says the length of a tsvector must be less than 1 MB, which seems correct, because tsvectorin() checks "if (buflen > MAXSTRPOS)” after the final data length has been calculated.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Residual tsquery cleanups
@ 2026-08-17 14:46 Tom Lane <tgl@sss.pgh.pa.us>
parent: Chao Li <li.evan.chao@gmail.com>
0 siblings, 0 replies; 3+ messages in thread
From: Tom Lane @ 2026-08-17 14:46 UTC (permalink / raw)
To: Chao Li <li.evan.chao@gmail.com>; +Cc: pgsql-hackers@lists.postgresql.org
Chao Li <li.evan.chao@gmail.com> writes:
> Overall looks good to me. A few small comments:
> 1 - 0001
> ```
> + unsigned int dist = pq_getmsgint(buf, sizeof(int16));
> +
> + if (dist > MAXENTRYPOS)
> + elog(ERROR, "invalid tsquery: invalid phrase distance %d",
> + dist);
> ```
> Here, “dist” is an unsigned int, so maybe it would be better to use “%u” than “%d” to print it.
Hmm ... doesn't really matter, since pq_getmsgint's result is
guaranteed to be in 0..65535. But OK.
> 2 - 0002
> ···
> + <para>The length of a <type>tsquery</type>'s data (lexemes only)
> + must be less than 1 megabyte</para>
> ···
> Look like to me this is inaccurate, because in pushValue_internal(), the "if (distance > MAXSTRPOS)” check happens before adding a new element, thus distance may exceed MAXSTRPOS after the element is added.
True, it can be up to one lexeme more. But I didn't think it was
worth trying to go into that much detail. As a comparison point,
the corresponding bullet for tsvector doesn't explain how to calculate
the space used for lexeme positions. I think the main takeaway here
is just that there *is* a limit of this type. People who really want
to know the exact details can consult the source code.
regards, tom lane
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2026-08-17 14:46 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-08-16 16:21 Residual tsquery cleanups Tom Lane <tgl@sss.pgh.pa.us>
2026-08-17 02:39 ` Chao Li <li.evan.chao@gmail.com>
2026-08-17 14:46 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox