Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xEChV-00000000z3L-1rtI for pgsql-bugs@arkaria.postgresql.org; Tue, 06 Oct 2026 21:27:53 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xEChU-000000023kP-0Nm5 for pgsql-bugs@arkaria.postgresql.org; Tue, 06 Oct 2026 21:27:52 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xEChT-000000023kG-3VN0 for pgsql-bugs@lists.postgresql.org; Tue, 06 Oct 2026 21:27:51 +0000 Received: from sss.pgh.pa.us ([68.162.161.243]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xEChQ-00000000iIG-49Sn for pgsql-bugs@lists.postgresql.org; Tue, 06 Oct 2026 21:27:50 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.18.1/8.18.1) with ESMTP id 696LRlns1568031; Tue, 6 Oct 2026 17:27:47 -0400 From: Tom Lane To: kehan5800@gmail.com cc: pgsql-bugs@lists.postgresql.org Subject: Re: BUG #19748: tsvectorrecv accepts duplicate lexemes and position 0 that tsvectorin rejects In-reply-to: <1091610.1791226091@sss.pgh.pa.us> References: <19748-6075f0508fa1ffc0@postgresql.org> <1091610.1791226091@sss.pgh.pa.us> Comments: In-reply-to Tom Lane message dated "Mon, 05 Oct 2026 14:48:11 -0400" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0" Content-ID: <1568007.1791322050.0@sss.pgh.pa.us> Date: Tue, 06 Oct 2026 17:27:47 -0400 Message-ID: <1568030.1791322067@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk ------- =_aaaaaaaaaa0 Content-Type: text/plain; charset="us-ascii" Content-ID: <1568007.1791322050.1@sss.pgh.pa.us> I wrote: > PG Bug reporting form writes: >> Two checks are still missing relative to tsvectorin(): >> 1. Duplicate lexemes are not merged. > So my proposal is to just throw an error if the lexeme sequence isn't > strictly increasing, as it already does for positions. That is > probably not something to back-patch, but this issue is surely not > important enough to worry about back-patching. >> 2. ... a leading position 0 is accepted. > Agreed, that's an oversight. Here's what that would look like. regards, tom lane ------- =_aaaaaaaaaa0 Content-Type: text/x-diff; name="v1-0001-Tighten-data-validity-checks-in-tsvectorrecv.patch"; charset="us-ascii" Content-ID: <1568007.1791322050.2@sss.pgh.pa.us> Content-Description: v1-0001-Tighten-data-validity-checks-in-tsvectorrecv.patch Content-Transfer-Encoding: quoted-printable =46rom 3c32c2cefc572489300d6fab5690e76a0e76cfd4 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Tue, 6 Oct 2026 17:23:45 -0400 Subject: [PATCH v1] Tighten data validity checks in tsvectorrecv(). Insist that the input lexemes be already sorted and non-duplicate. There's no good reason for them not to be, since the required ordering is simple and platform-independent. The previous coding was willing to re-sort them, but that left a gap: it didn't de-duplicate them. De-duplicating would potentially require merging position lists, which would be painful; we can't easily re-use tsvectorin()'s logic for that, since it uses a different temporary representation. Let's just dodge the problem by insisting the data be fully correct as-presented, as we were already doing for the position lists. Also, there was an oversight in the position-list checking: it did not verify that the first position isn't zero. While these problems allow tsvectorrecv() to construct tsvectors that would not be generated internally, the vectors aren't so broken that we can't work with them. Therefore, the best course of action seems to be to apply these tighter rules in master but not back-patch. Back-patching might break application workflows that aren't causing any serious problems. Any such users will need to clean their data before loading it into PG >=3D v20, but they'd face that issue anyway, and a major update is a better time for it than a minor update. Bug: #19748 Reported-by: Ke Author: Tom Lane Discussion: https://postgr.es/m/19748-6075f0508fa1ffc0@postgresql.org --- src/backend/utils/adt/tsvector.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsve= ctor.c index 21d29a7e4f6..2fe782374e7 100644 --- a/src/backend/utils/adt/tsvector.c +++ b/src/backend/utils/adt/tsvector.c @@ -454,7 +454,6 @@ tsvectorrecv(PG_FUNCTION_ARGS) * WordEntries */ Size hdrlen; Size len; /* allocated size of vec */ - bool needSort =3D false; = nentries =3D pq_getmsgint(buf, sizeof(int32)); = @@ -512,16 +511,18 @@ tsvectorrecv(PG_FUNCTION_ARGS) = datalen +=3D lex_len; = + /* Verify correct sorting of lexemes, with no duplicates */ if (i > 0 && compareentry(&vec->entries[i], &vec->entries[i - 1], STRPTR(vec)) <=3D 0) - needSort =3D true; + elog(ERROR, "tsvector lexemes are misordered"); = /* Receive positions */ if (npos > 0) { uint16 j; WordEntryPos *wepptr; + WordEntryPos lastpos =3D 0; = /* * Pad to 2-byte alignment if necessary. Though we used palloc0 @@ -539,9 +540,13 @@ tsvectorrecv(PG_FUNCTION_ARGS) wepptr =3D POSDATAPTR(vec, &vec->entries[i]); for (j =3D 0; j < npos; j++) { - wepptr[j] =3D (WordEntryPos) pq_getmsgint(buf, sizeof(WordEntryPos)); - if (j > 0 && WEP_GETPOS(wepptr[j]) <=3D WEP_GETPOS(wepptr[j - 1])) + WordEntryPos thispos; + + thispos =3D (WordEntryPos) pq_getmsgint(buf, sizeof(WordEntryPos)); + /* Verify positions are sorted, nonduplicate, and not zero */ + if (WEP_GETPOS(thispos) <=3D WEP_GETPOS(lastpos)) elog(ERROR, "position information is misordered"); + wepptr[j] =3D lastpos =3D thispos; } = datalen +=3D sizeof(uint16) + npos * sizeof(WordEntryPos); @@ -559,9 +564,5 @@ tsvectorrecv(PG_FUNCTION_ARGS) = SET_VARSIZE(vec, hdrlen + datalen); = - if (needSort) - qsort_arg(ARRPTR(vec), vec->size, sizeof(WordEntry), - compareentry, STRPTR(vec)); - PG_RETURN_TSVECTOR(vec); } -- = 2.52.0 ------- =_aaaaaaaaaa0--