From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: uwe.binder@pass-consulting.com
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18080: to_tsvector fails for long text input
Date: Fri, 22 Sep 2023 13:48:59 -0400
Message-ID: <1056584.1695404939@sss.pgh.pa.us> (raw)
In-Reply-To: <3300287.1694786029@sss.pgh.pa.us>
References: <202309151141.pq2zpi5kxdvn@alvherre.pgsql>
<3300287.1694786029@sss.pgh.pa.us>
I wrote:
> Yeah. My thought about blocking the error had been to limit
> prs.lenwords to MaxAllocSize/sizeof(ParsedWord) in this code.
Concretely, as attached. This allows the given test case to
complete, since it doesn't actually create very many distinct
words. In other cases we could expect to fail when the array
has to get enlarged, but that's just a normal implementation
limitation.
I looked for other places that might initialize lenwords
to not-sane values, and didn't find any.
BTW, the field order in ParsedWord is such that there's a fair
amount of wasted pad space on 64-bit builds. I doubt we can
get away with rearranging it in released branches; but maybe
it's worth doing something about that in HEAD, to push out
the point at which you hit the 1Gb limit.
regards, tom lane
Attachments:
[text/x-diff] bound-lenwords-in-to_tsvector_byid.patch (533B, ../1056584.1695404939@sss.pgh.pa.us/2-bound-lenwords-in-to_tsvector_byid.patch)
download | inline diff:diff --git a/src/backend/tsearch/to_tsany.c b/src/backend/tsearch/to_tsany.cindex 3b6d41f9e8..fe39d6c4b9 100644--- a/src/backend/tsearch/to_tsany.c+++ b/src/backend/tsearch/to_tsany.c@@ -252,6 +252,8 @@ to_tsvector_byid(PG_FUNCTION_ARGS)
* number */
if (prs.lenwords < 2)
prs.lenwords = 2;
+ else if (prs.lenwords > MaxAllocSize / sizeof(ParsedWord))+ prs.lenwords = MaxAllocSize / sizeof(ParsedWord);
prs.curwords = 0;
prs.pos = 0;
prs.words = (ParsedWord *) palloc(sizeof(ParsedWord) * prs.lenwords);
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: tgl@sss.pgh.pa.us, alvherre@alvh.no-ip.org, uwe.binder@pass-consulting.com, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18080: to_tsvector fails for long text input
In-Reply-To: <1056584.1695404939@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox