public inbox for [email protected]  

help / color / mirror / Atom feed

From: Jose Luis Tallon <[email protected]>
To: Maxim Zibitsker <[email protected]>
To: [email protected]
Subject: Re: Support allocating memory for large strings
Date: Sat, 8 Nov 2025 12:17:19 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>

On 8/11/25 3:15, Maxim Zibitsker wrote:
> PostgreSQL's MaxAllocSize limit prevents storing individual variable-length character strings exceeding ~1GB, causing "invalid memory alloc request size" errors during INSERT operations on tables with large text columns. Example reproduction included in artifacts.md.

Tom Lane's very appropriate response not withstanding....

a) Why is this a problem? (Please share a bit more about your intended 
use case)

b) Why would someone need to store >1GB worth of TEXT (in a single 
string, no less!) in a column in an (albeit very flexible) Relational 
Database ?

     (I'm assuming no internal structure that would allow such amount of 
text to be split/spread over multiple records)

c) There exists LObs (Large OBjects) intended for this use, precisely... 
why is this mechanism not a good solution to your need?

d) Wouldn't a (journalling) File System (with a slim abstraction layer 
on top for directory hashing/indexing) not be a better solution for this 
particular application?

     Full Text Search on the stored data doesn't look like it would ever 
be performant... there exist specialized tools for that


And... how did you get "invalid" data in the database, that pg_dump 
wouldn't process, in the first place? (maybe just speculating/projecting 
and I didn't pick up the nuance properly)


Mostly curious about the problem / intended use case.... when we 
explored limits and limitations in Postgres almost 15 years ago, we 
never considered this even :o



Thanks,

-- 
Parkinson's Law: Work expands to fill the time alloted to it.

view thread (5+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: Support allocating memory for large strings
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox