To: "Tom Lane" <tgl@sss.pgh.pa.us>
Cc: "Andreas Joseph Krogh" <andreak@officenet.no>, <pgsql-sql@postgresql.org>
Subject: Re: Returning the total number of rows as a separate column when
	using limit
In-Reply-To: <19503.1194274853@sss.pgh.pa.us> (Tom Lane's message of "Mon\,
	05 Nov 2007 10\:00\:53 -0500")
User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1 (gnu/linux)
References: <200711050836.38130.andreak@officenet.no>
	<18728.1194272302@sss.pgh.pa.us>
	<200711051538.34536.andreak@officenet.no>
	<19503.1194274853@sss.pgh.pa.us>
From: Gregory Stark <stark@enterprisedb.com>
Organization: EnterpriseDB
Date: Mon, 05 Nov 2007 15:27:03 +0000
Message-ID: <87ir4g67co.fsf@oxford.xeocode.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii


> Andreas Joseph Krogh <andreak@officenet.no> writes:
>> On Monday 05 November 2007 15:18:22 Tom Lane wrote:
>>> That's only an estimate.  Since the query doesn't get executed to
>>> completion thanks to the LIMIT, Postgres really has no idea whether
>>> the estimate is accurate.
>
>> Ok. The query is ORDER-ed, but you're saying that it doesn't matter and PG 
>> still doesn't have to know the total numbers even if it has to sort the 
>> result?

At a guess you're displaying pages of information and want to display
something like "displaying 1-10 of 150" ?

Postgres is kind of lacking a solution for this problem. The last time I
needed to do this I bit the bullet and ran the query twice, once with a
"select count(*) from (...)" around it and once with "select * from (...)
order by x offset n limit m" around it. The second time runs relatively
quickly since all the raw data is in cache.

The "right" way to do this would probably be to have a temporary table which
you populate in one step, perform the count(*) on in a second query, then
select the page of data with the ordering in a third query. Then you can keep
the data around for some limited amount of time in case the user accesses a
second page. But this requires some infrastructure to keep track of the cached
data and what query it corresponded to and determine when to replace it with
new data or drop it.

However Postgres temporary tables are fairly expensive and if you're creating
them for every web access you're going to have to vacuum the system catalogs
quite frequently. They're not really well suited for this task.

Alternatively you could create a cursor and play with that. But I don't think
that's a great solution either. (yet? I think cursors are getting more useful
in Postgres, perhaps it will be eventually.)

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com
  Ask me about EnterpriseDB's RemoteDBA services!