From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: allocation limit for encoding conversion
Date: Tue, 24 Sep 2019 18:09:28 -0400
Message-ID: <5884.1569362968@sss.pgh.pa.us> (raw)
In-Reply-To: <20190924214204.mav4md77xg5u5wq6@alap3.anarazel.de>
References: <20190816181418.GA898@alvherre.pgsql>
<27990.1565991109@sss.pgh.pa.us>
<20190816220452.lg57jowanbsa2gf7@alap3.anarazel.de>
<974.1569356381@sss.pgh.pa.us>
<20190924214204.mav4md77xg5u5wq6@alap3.anarazel.de>
Andres Freund <andres@anarazel.de> writes:
> On 2019-09-24 16:19:41 -0400, Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> It's probably too big a hammer for this specific case, but I think at
>>> some point we ought to stop using fixed size allocations for this kind
>>> of work. Instead we should use something roughly like our StringInfo,
>>> except that when exceeding the current size limit, the overflowing data
>>> is stored in a separate allocation. And only once we actually need the
>>> data in a consecutive form, we allocate memory that's large enough to
>>> store the all the separate allocations in their entirety.
>> That sounds pretty messy :-(.
> I don't think it's too bad - except for now allowing the .data member of
> such a 'chunked' StringInfo to be directly accessible, it'd be just
> about the same interface as the current stringinfo.
I dunno. What you're describing would be a whole lotta work, and it'd
break a user-visible API, and no amount of finagling is going to prevent
it from making conversions somewhat slower, and the cases where it matters
to not preallocate a surely-large-enough buffer are really few and far
between. I have to think that we have better ways to spend our time.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tgl@sss.pgh.pa.us, andres@anarazel.de, alvherre@2ndquadrant.com, pgsql-hackers@lists.postgresql.org
Subject: Re: allocation limit for encoding conversion
In-Reply-To: <5884.1569362968@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox