X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 9B8443A455D for ; Mon, 1 Nov 2004 13:46:27 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 27243-07 for ; Mon, 1 Nov 2004 13:46:22 +0000 (GMT) Received: from linuxworld.com.au (unknown [203.34.46.50]) by svr1.postgresql.org (Postfix) with ESMTP id 3E1C43A455C for ; Mon, 1 Nov 2004 13:46:23 +0000 (GMT) Received: from localhost (swm@localhost) by linuxworld.com.au (8.11.6/8.11.4) with ESMTP id iA1Dj7e05430; Tue, 2 Nov 2004 00:45:10 +1100 Date: Tue, 2 Nov 2004 00:45:07 +1100 (EST) From: Gavin Sherry X-X-Sender: swm@linuxworld.com.au To: Dustin Sallings Cc: TTK Ciar , pgsql-performance@postgresql.org Subject: Re: psql large RSS (1.6GB) In-Reply-To: <8132EBC8-2B02-11D9-A2DE-000A957659CC@spy.net> Message-ID: References: <20041027075748.GA8355@hardpoint.ciar.org> <8132EBC8-2B02-11D9-A2DE-000A957659CC@spy.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200411/3 X-Sequence-Number: 8957 On Sat, 30 Oct 2004, Dustin Sallings wrote: > > If the solution is to just write a little client that uses perl > > DBI to fetch rows one at a time and write them out, that's doable, > > but it would be nice if psql could be made to "just work" without > > the monster RSS. > > It wouldn't make a difference unless that driver implements the > underlying protocol on its own. Even though we can tell people to make use of cursors, it seems that memory usage for large result sets should be addressed. A quick search of the archives does not reveal any discussion about having libpq spill to disk if a result set reaches some threshold. Has this been canvassed in the past? Thanks, Gavin