From: PG Bug reporting form <noreply@postgresql.org>
To: pgsql-bugs@lists.postgresql.org
Cc: magicagent@gmail.com
Subject: BUG #17557: ts_headline will error with "invalid memory alloc request size" for large documents
Date: Fri, 22 Jul 2022 15:39:42 +0000
Message-ID: <17557-6ddc074c8b1bd6df@postgresql.org> (raw)
The following bug has been logged on the website:
Bug reference: 17557
Logged by: Alex Malek
Email address: magicagent@gmail.com
PostgreSQL version: 14.4
Operating system: Red Hat
Description:
ts_headline when given a documents over a certain size/number of words will
cause "ERROR: invalid memory alloc request size XXXXXX"
# select ts_headline('b ' || repeat('1 ',16777215), $$'b'$$::tsquery,
'MaxWords=4, MinWords=3') ;
ERROR: invalid memory alloc request size 1610612736
Not just related to document size but also to number of "words" in a
document:
One less "word" works:
select ts_headline('b ' || repeat('1 ',16777214), $$'b'$$::tsquery,
'MaxWords=4, MinWords=3') ;
ts_headline
----------------
<b>b</b> 1 1 1
(1 row)
Mem not an issue for larger "words" up to a point:
# select ts_headline('b ' || repeat('123456789012345 ',16777214),
$$'b'$$::tsquery, 'MaxWords=4, MinWords=3') ;
ts_headline
----------------------------------------------------------
<b>b</b> 123456789012345 123456789012345 123456789012345
(1 row)
# select ts_headline('b ' || repeat('1234567890123456 ',16777214),
$$'b'$$::tsquery, 'MaxWords=4, MinWords=3') ;
ERROR: invalid memory alloc request size 1140850564
Memory issue appears to be triggered based on total number of words and word
length
# select ts_headline('b ' || repeat('1234567890123456 ',15790000),
$$'b'$$::tsquery, 'MaxWords=4, MinWords=3') ;
ts_headline
-------------------------------------------------------------
<b>b</b> 1234567890123456 1234567890123456 1234567890123456
(1 row)
# select ts_headline('b ' || repeat('1234567890123456 ',15795000),
$$'b'$$::tsquery, 'MaxWords=4, MinWords=3') ;
ERROR: invalid memory alloc request size 1074060012
I get the same results even when increasing psql GUCs including work_mem,
shared_buffers and effective_cache_size
Also on machines w/ significantly more RAM, with and w/o HugePages enabled.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: noreply@postgresql.org, pgsql-bugs@lists.postgresql.org, magicagent@gmail.com
Subject: Re: BUG #17557: ts_headline will error with "invalid memory alloc request size" for large documents
In-Reply-To: <17557-6ddc074c8b1bd6df@postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox