From: Tom Lane <tgl@sss.pgh.pa.us>
To: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
cc: =?UTF-8?B?0JXQs9C+0YAg0KfQuNC90LTRj9GB0LrQuNC9?= <kyzevan23@mail.ru>,
        Richard Guo <guofenglinux@gmail.com>,
        Alvaro Herrera <alvherre@alvh.no-ip.org>,
        mahendrakar s <mahendrakarforpg@gmail.com>
Subject: Re: Stack overflow issue
In-reply-to: <1120159.1661362215@sss.pgh.pa.us>
References: 
 <CABkiuWrDt75d1pFaahQ0AsqYxqx3AQRJ+ajd1Wv=CFTW8ZSJFQ@mail.gmail.com>
 <20220824104953.zxmciclbdn3dnb2g@alvherre.pgsql>
 <CAMbWs4_p9rP=0XtVqzZf6dEFx0MHmPFOP4PhG8N8=_tqk2+yjA@mail.gmail.com>
 <CAMbWs48iswb9cXXnq1Ocy0WLdov__1eR=qenRP+a3x1PJiphJg@mail.gmail.com>
 <1014604.1661349796@sss.pgh.pa.us> <1120159.1661362215@sss.pgh.pa.us>
Comments: In-reply-to Tom Lane <tgl@sss.pgh.pa.us>
	message dated "Wed, 24 Aug 2022 13:30:15 -0400"
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0"
Content-ID: <3661058.1661871698.0@sss.pgh.pa.us>
Date: Tue, 30 Aug 2022 11:02:38 -0400
Message-ID: <3661156.1661871758@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/3661156.1661871758%40sss.pgh.pa.us>
Precedence: bulk

------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <3661058.1661871698.1@sss.pgh.pa.us>

I wrote:
>> I think most likely we should report this to Snowball upstream
>> and see what they think is an appropriate fix.

> Done at [1], and I pushed the other fixes.  Thanks again for the report!

The upstream recommendation, which seems pretty sane to me, is to
simply reject any string exceeding some threshold length as not
possibly being a word.  Apparently it's common to use thresholds
as small as 64 bytes, but in the attached I used 1000 bytes.

			regards, tom lane


------- =_aaaaaaaaaa0
Content-Type: text/x-diff;
	name="limit-length-of-strings-passed-to-snowball.patch";
	charset="us-ascii"
Content-ID: <3661058.1661871698.2@sss.pgh.pa.us>
Content-Description: limit-length-of-strings-passed-to-snowball.patch
Content-Transfer-Encoding: quoted-printable

diff --git a/src/backend/snowball/dict_snowball.c b/src/backend/snowball/d=
ict_snowball.c
index 68c9213f69..aaf4ff72b6 100644
--- a/src/backend/snowball/dict_snowball.c
+++ b/src/backend/snowball/dict_snowball.c
@@ -272,11 +272,25 @@ dsnowball_lexize(PG_FUNCTION_ARGS)
 	DictSnowball *d =3D (DictSnowball *) PG_GETARG_POINTER(0);
 	char	   *in =3D (char *) PG_GETARG_POINTER(1);
 	int32		len =3D PG_GETARG_INT32(2);
-	char	   *txt =3D lowerstr_with_len(in, len);
 	TSLexeme   *res =3D palloc0(sizeof(TSLexeme) * 2);
+	char	   *txt;
 =

+	/*
+	 * Reject strings exceeding 1000 bytes, as they're surely not words in a=
ny
+	 * human language.  This restriction avoids wasting cycles on stuff like
+	 * base64-encoded data, and it protects us against possible inefficiency
+	 * or misbehavior in the stemmers (for example, the Turkish stemmer has =
an
+	 * indefinite recursion so it can crash on long-enough strings).
+	 */
+	if (len <=3D 0 || len > 1000)
+		PG_RETURN_POINTER(res);
+
+	txt =3D lowerstr_with_len(in, len);
+
+	/* txt is probably not zero-length now, but we'll check anyway */
 	if (*txt =3D=3D '\0' || searchstoplist(&(d->stoplist), txt))
 	{
+		/* empty or stopword, so reject */
 		pfree(txt);
 	}
 	else

------- =_aaaaaaaaaa0--