X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 0347C9FA30C for ; Mon, 5 Jun 2006 13:45:35 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 41833-04 for ; Mon, 5 Jun 2006 13:45:24 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from linux.dunaweb.hu (linux.dunaweb.hu [62.77.196.1]) by postgresql.org (Postfix) with ESMTP id 457819FA12F for ; Mon, 5 Jun 2006 13:45:21 -0300 (ADT) Received: from localhost (linux.dunaweb.hu [127.0.0.1]) by linux.dunaweb.hu (Postfix) with ESMTP id 6C06B8E0895; Mon, 5 Jun 2006 18:45:19 +0200 (CEST) Received: from linux.dunaweb.hu ([127.0.0.1]) by localhost (linux.dunaweb.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 23503-08; Mon, 5 Jun 2006 18:45:12 +0200 (CEST) Received: from [81.17.177.202] (host-81-17-177-202.dunaweb.hu [81.17.177.202]) by linux.dunaweb.hu (Postfix) with ESMTP id ED5C88E0804; Mon, 5 Jun 2006 18:45:11 +0200 (CEST) Message-ID: <44845F99.7050306@dunaweb.hu> Date: Mon, 05 Jun 2006 18:45:13 +0200 From: Zoltan Boszormenyi User-Agent: Thunderbird 1.5.0.2 (X11/20060501) MIME-Version: 1.0 To: Tom Lane Cc: "Jim C. Nasby" , pgsql-hackers@postgresql.org Subject: Re: [PERFORM] psql -A (unaligned format) eats too much References: <44835834.2020309@dunaweb.hu> <44835F86.5040008@dunaweb.hu> <20060605152258.GT53487@pervasive.com> <20512.1149521250@sss.pgh.pa.us> In-Reply-To: <20512.1149521250@sss.pgh.pa.us> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: amavisd-new at dunaweb.hu X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200606/163 X-Sequence-Number: 84396 Hi! Tom Lane írta: > "Jim C. Nasby" writes: > >> I've been able to verify this on 8.1.4; psql -A -t -c 'SELECT * FROM >> largetable' > /dev/null results in psql consuming vast quantities of >> memory. Why is this? >> > > Is it different without the -A? > > I'm reading this as just another uninformed complaint about libpq's > habit of buffering the whole query result. It's possible that there's > a memory leak in the -A path specifically, but nothing said so far > provided any evidence for that. > > regards, tom lane > So, is libpq always buffering the result? Thanks. I thought psql buffers only because in its formatted output mode it has to know the widest value for all the columns. Then the SELECT INTO TEMP ; COPY TO STDOUT solution I found is _the_ solution. I guess then the libpq-based ODBC driver suffers from the same problem? It certainly explains the performance problems I observed: the server finishes the query, the ODBC driver (or libpq underneath) fetches all the records and the application receives the first record after all these. Nice. Best regards, Zoltán Böszörményi