From: Nathan Bossart <nathandbossart@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Maxim Zibitsker <max.zibitsker@gmail.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Support allocating memory for large strings
Date: Mon, 10 Nov 2025 14:35:09 -0600
Message-ID: <aRJMffw0OEScpr3F@nathan> (raw)
In-Reply-To: <1357347.1762569165@sss.pgh.pa.us>
References: <5B146D62-35AE-4822-9B98-37E386AFB942@gmail.com>
<1357347.1762569165@sss.pgh.pa.us>
On Fri, Nov 07, 2025 at 09:32:45PM -0500, Tom Lane wrote:
> Maxim Zibitsker <max.zibitsker@gmail.com> writes:
>> PostgreSQL's MaxAllocSize limit prevents storing individual
>> variable-length character strings exceeding ~1GB, causing "invalid
>> memory alloc request size" errors during INSERT operations on tables
>> with large text columns.
>
> This is news to no one. We are not especially interested in trying to
> relax that limit, because doing so would bleed over into approximately
> everything in the backend, and create opportunities for
> integer-overflow bugs in many places that are perfectly okay today.
> The cost-benefit ratio for changing this decision is horrible.
FWIW something I am hearing about more often these days, and what I believe
Maxim's patch is actually after, is the 1GB limit on row size. Even if
each field doesn't exceed 1GB (which is what artifacts.md seems to
demonstrate), heap_form_tuple() and friends can fail to construct the whole
tuple. This doesn't seem to be covered in the existing documentation about
limits [0].
[0] https://www.postgresql.org/docs/devel/limits.html
--
nathan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, tgl@sss.pgh.pa.us, max.zibitsker@gmail.com
Subject: Re: Support allocating memory for large strings
In-Reply-To: <aRJMffw0OEScpr3F@nathan>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox