Date: Mon, 10 Nov 2025 14:35:09 -0600
From: Nathan Bossart <nathandbossart@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Maxim Zibitsker <max.zibitsker@gmail.com>, pgsql-hackers@postgresql.org
Subject: Re: Support allocating memory for large strings
Message-ID: <aRJMffw0OEScpr3F@nathan>
References: <5B146D62-35AE-4822-9B98-37E386AFB942@gmail.com>
 <1357347.1762569165@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1357347.1762569165@sss.pgh.pa.us>
Archived-At: <https://www.postgresql.org/message-id/aRJMffw0OEScpr3F%40nathan>
Precedence: bulk

On Fri, Nov 07, 2025 at 09:32:45PM -0500, Tom Lane wrote:
> Maxim Zibitsker <max.zibitsker@gmail.com> writes:
>> PostgreSQL's MaxAllocSize limit prevents storing individual
>> variable-length character strings exceeding ~1GB, causing "invalid
>> memory alloc request size" errors during INSERT operations on tables
>> with large text columns.
> 
> This is news to no one.  We are not especially interested in trying to
> relax that limit, because doing so would bleed over into approximately
> everything in the backend, and create opportunities for
> integer-overflow bugs in many places that are perfectly okay today.
> The cost-benefit ratio for changing this decision is horrible.

FWIW something I am hearing about more often these days, and what I believe
Maxim's patch is actually after, is the 1GB limit on row size.  Even if
each field doesn't exceed 1GB (which is what artifacts.md seems to
demonstrate), heap_form_tuple() and friends can fail to construct the whole
tuple.  This doesn't seem to be covered in the existing documentation about
limits [0].

[0] https://www.postgresql.org/docs/devel/limits.html

-- 
nathan