public inbox for [email protected]
help / color / mirror / Atom feedFrom: Tom Lane <[email protected]>
To: Maxim Zibitsker <[email protected]>
Cc: [email protected]
Subject: Re: Support allocating memory for large strings
Date: Fri, 07 Nov 2025 21:32:45 -0500
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
Maxim Zibitsker <[email protected]> writes:
> PostgreSQL's MaxAllocSize limit prevents storing individual variable-length character strings exceeding ~1GB, causing "invalid memory alloc request size" errors during INSERT operations on tables with large text columns.
This is news to no one. We are not especially interested in trying to
relax that limit, because doing so would bleed over into approximately
everything in the backend, and create opportunities for
integer-overflow bugs in many places that are perfectly okay today.
The cost-benefit ratio for changing this decision is horrible.
> The attached patches demonstrates a proof of concept using
> palloc_extended with MCXT_ALLOC_HUGE in the write path.
"Proof of concept"? This can't possibly fix your problem, because it
does nothing for the fact that tuple size fields are still limited
to 1GB, as are varlena headers for individual fields. A serious
attack on this limitation, at a guess, would require a patch on the
order of 100K lines, and that might be an underestimate.
regards, tom lane
view thread (5+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: Support allocating memory for large strings
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox