From: Zoltan Boszormenyi <zboszor@dunaweb.hu>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Mark Woodward <pgsql@mohawksoft.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Jim C. Nasby <jnasby@pervasive.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
Date: Mon, 05 Jun 2006 19:17:31 +0200
Message-ID: <4484672B.8080606@dunaweb.hu> (raw)
In-Reply-To: <44845E86.3080001@dunslane.net>
References: <44835834.2020309@dunaweb.hu>
<44835F86.5040008@dunaweb.hu>
<20060605152258.GT53487@pervasive.com>
<20512.1149521250@sss.pgh.pa.us>
<20060605155349.GU53487@pervasive.com>
<20813.1149523235@sss.pgh.pa.us>
<18006.24.91.171.78.1149526092.squirrel@mail.mohawksoft.com>
<44845E86.3080001@dunslane.net>
Andrew Dunstan írta:
> Mark Woodward wrote:
>>> "Jim C. Nasby" <jnasby@pervasive.com> writes:
>>>
>>>> On Mon, Jun 05, 2006 at 11:27:30AM -0400, Tom Lane wrote:
>>>>
>>>>> I'm reading this as just another uninformed complaint about libpq's
>>>>> habit of buffering the whole query result. It's possible that
>>>>> there's
>>>>> a memory leak in the -A path specifically, but nothing said so far
>>>>> provided any evidence for that.
>>>>>
>>>> Certainly seems like it. It seems like it would be good to allow for
>>>> libpq not to buffer, since there's cases where it's not needed...
>>>>
>>> See past discussions. The problem is that libpq's API says that
>>> when it
>>> hands you back the completed query result, the command is complete and
>>> guaranteed not to fail later. A streaming interface could not make
>>> that
>>> guarantee, so it's not a transparent substitution.
>>>
>>> I wouldn't have any strong objection to providing a separate API that
>>> operates in a streaming fashion, but defining it is something no one's
>>> bothered to do yet. In practice, if you have to code to a variant API,
>>> it's not that much more trouble to use a cursor...
>>>
>>>
>>
>> Wouldn't the "COPY (select ...) TO STDOUT" format being discussed solve
>> this for free?
Yes, it would for me.
> It won't solve it in the general case for clients that expect a result
> set. ISTM that "use a cursor" is a perfectly reasonable answer, though.
The general case cannot be applied for all particular cases.
E.g. you cannot use cursors from shell scripts and just for
producing an "export file" it's not too reasonable either.
Redirecting psql's output or COPY is enough.
Best regards,
Zoltán Böszörényi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: zboszor@dunaweb.hu, andrew@dunslane.net, pgsql@mohawksoft.com, tgl@sss.pgh.pa.us, jnasby@pervasive.com, pgsql-hackers@postgresql.org
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
In-Reply-To: <4484672B.8080606@dunaweb.hu>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox