X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id AA2B29FA6B1 for ; Mon, 5 Jun 2006 13:00:57 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 34410-02 for ; Mon, 5 Jun 2006 13:00:42 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by postgresql.org (Postfix) with ESMTP id B84509FA2FD for ; Mon, 5 Jun 2006 13:00:41 -0300 (ADT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id k55G0Z7D020814; Mon, 5 Jun 2006 12:00:35 -0400 (EDT) To: "Jim C. Nasby" cc: Zoltan Boszormenyi , pgsql-hackers@postgresql.org Subject: Re: [PERFORM] psql -A (unaligned format) eats too much memory In-reply-to: <20060605155349.GU53487@pervasive.com> References: <44835834.2020309@dunaweb.hu> <44835F86.5040008@dunaweb.hu> <20060605152258.GT53487@pervasive.com> <20512.1149521250@sss.pgh.pa.us> <20060605155349.GU53487@pervasive.com> Comments: In-reply-to "Jim C. Nasby" message dated "Mon, 05 Jun 2006 10:53:49 -0500" Date: Mon, 05 Jun 2006 12:00:35 -0400 Message-ID: <20813.1149523235@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200606/160 X-Sequence-Number: 84393 "Jim C. Nasby" writes: > On Mon, Jun 05, 2006 at 11:27:30AM -0400, Tom Lane wrote: >> I'm reading this as just another uninformed complaint about libpq's >> habit of buffering the whole query result. It's possible that there's >> a memory leak in the -A path specifically, but nothing said so far >> provided any evidence for that. > Certainly seems like it. It seems like it would be good to allow for > libpq not to buffer, since there's cases where it's not needed... See past discussions. The problem is that libpq's API says that when it hands you back the completed query result, the command is complete and guaranteed not to fail later. A streaming interface could not make that guarantee, so it's not a transparent substitution. I wouldn't have any strong objection to providing a separate API that operates in a streaming fashion, but defining it is something no one's bothered to do yet. In practice, if you have to code to a variant API, it's not that much more trouble to use a cursor... regards, tom lane