From: Tom Lane <tgl@sss.pgh.pa.us>
To: =?koi8-r?B?5dLPyMnOIOTFzsnTIPfMwcTJzcnSz9fJ3g==?= <erohin-d@datagile.ru>
cc: pgsql-hackers@postgresql.org
Subject: Re: Fix receiving large legal tsvector from binary format
In-reply-to: <009801d9f2d9$f29730c0$d7c59240$@datagile.ru>
References: <009801d9f2d9$f29730c0$d7c59240$@datagile.ru>
Comments: In-reply-to =?koi8-r?B?5dLPyMnOIOTFzsnTIPfMwcTJzcnSz9fJ3g==?=
 <erohin-d@datagile.ru>
	message dated "Fri, 29 Sep 2023 13:36:27 -0000"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1842216.1696181041.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Sun, 01 Oct 2023 13:24:01 -0400
Message-ID: <1842217.1696181041@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/1842217.1696181041%40sss.pgh.pa.us>
Precedence: bulk

=3D?koi8-r?B?5dLPyMnOIOTFzsnTIPfMwcTJzcnSz9fJ3g=3D=3D?=3D <erohin-d@datagi=
le.ru> writes:
> There is a problem on receiving large tsvector from binary format with
> getting error "invalid tsvector: maximum total lexeme length exceeded".

Good catch!  Even without an actual failure, we'd be wasting space
on-disk anytime we stored a tsvector received through binary input.

I pushed your 0001 and 0002, but I don't really agree that 0003
is an improvement.  It looks to me like it'll result in one
repalloc per lexeme, instead of the O(log N) behavior we had before.
It's not that important to keep the palloc chunk size small here,
given that we don't allow tsvectors to get anywhere near 1Gb anyway.

			regards, tom lane