Received: from malur.postgresql.org ([217.196.149.56])
	by arkaria.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <pgsql-bugs-owner+archive@lists.postgresql.org>)
	id 1qh9Gf-001U7A-8u
	for pgsql-bugs@arkaria.postgresql.org; Fri, 15 Sep 2023 13:53:57 +0000
Received: from localhost ([127.0.0.1] helo=malur.postgresql.org)
	by malur.postgresql.org with esmtp (Exim 4.94.2)
	(envelope-from <pgsql-bugs-owner+archive@lists.postgresql.org>)
	id 1qh9Gd-000EhF-L4
	for pgsql-bugs@arkaria.postgresql.org; Fri, 15 Sep 2023 13:53:55 +0000
Received: from makus.postgresql.org ([2001:4800:3e1:1::229])
	by malur.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <tgl@sss.pgh.pa.us>)
	id 1qh9Gd-000Egl-B8
	for pgsql-bugs@lists.postgresql.org; Fri, 15 Sep 2023 13:53:55 +0000
Received: from sss.pgh.pa.us ([68.162.161.243])
	by makus.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <tgl@sss.pgh.pa.us>)
	id 1qh9Ga-004jJf-FP
	for pgsql-bugs@lists.postgresql.org; Fri, 15 Sep 2023 13:53:54 +0000
Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1])
	by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 38FDrnBU3300288;
	Fri, 15 Sep 2023 09:53:49 -0400
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
cc: uwe.binder@pass-consulting.com, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18080: to_tsvector fails for long text input
In-reply-to: <202309151141.pq2zpi5kxdvn@alvherre.pgsql>
References: <202309151141.pq2zpi5kxdvn@alvherre.pgsql>
Comments: In-reply-to Alvaro Herrera <alvherre@alvh.no-ip.org>
	message dated "Fri, 15 Sep 2023 13:41:56 +0200"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <3300286.1694786029.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Fri, 15 Sep 2023 09:53:49 -0400
Message-ID: <3300287.1694786029@sss.pgh.pa.us>
List-Id: <pgsql-bugs.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-bugs@lists.postgresql.org>
List-Owner: <mailto:pgsql-bugs-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-bugs>
Archived-At: 
 <https://www.postgresql.org/message-id/3300287.1694786029%40sss.pgh.pa.us>
Precedence: bulk

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> On 2023-Sep-04, PG Bug reporting form wrote:
>> SELECT to_tsvector('english'::regconfig, (REPEAT('<Long123456789/>'::te=
xt,
>> 20000000)));
>> results in
>> ERROR:  invalid memory alloc request size 2133333320

> This is because to_tsvector_byid does this:
>     prs.lenwords =3D VARSIZE_ANY_EXHDR(in) / 6;    /* just estimation of=
 word's
>                                                  * number */
>     if (prs.lenwords < 2)
>         prs.lenwords =3D 2;

Yeah.  My thought about blocking the error had been to limit
prs.lenwords to MaxAllocSize/sizeof(ParsedWord) in this code.
I doubt that switching over to MCXT_ALLOC_HUGE is a good idea.
(Would we not also have to touch the places that repalloc that
array?)

			regards, tom lane