X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 8C22B3A5DAB for ; Thu, 27 Jan 2005 05:50:32 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 95451-01 for ; Thu, 27 Jan 2005 05:50:29 +0000 (GMT) Received: from floppy.pyrenet.fr (news.pyrenet.fr [194.250.190.2]) by svr1.postgresql.org (Postfix) with ESMTP id B43A03A5DF7 for ; Thu, 27 Jan 2005 05:50:29 +0000 (GMT) Received: by floppy.pyrenet.fr (Postfix, from userid 106) id EBEB6313E5; Thu, 27 Jan 2005 06:50:28 +0100 (MET) From: Christopher Browne X-Newsgroups: pgsql.performance Subject: Re: [SQL] OFFSET impact on Performance??? Date: Thu, 27 Jan 2005 00:10:24 -0500 Organization: cbbrowne Computing Inc Lines: 61 Message-ID: References: <6EE64EF3AB31D5448D0007DD34EEB3412A75D3@Herge.rcsinc.local> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Complaints-To: usenet@news.hub.org X-message-flag: Outlook is rather hackable, isn't it? X-Home-Page: http://www.cbbrowne.com/info/ X-Affero: http://svcs.affero.net/rm.php?r=cbbrowne User-Agent: Gnus/5.1007 (Gnus v5.10.7) XEmacs/21.4 (Corporate Culture, linux) Cancel-Lock: sha1:/hkjkmWSoNhoRHZXrjXWpcQDbWk= To: pgsql-performance@postgresql.org X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.077 tagged_above=0 required=5 tests=AWL X-Spam-Level: X-Archive-Number: 200501/524 X-Sequence-Number: 10214 In an attempt to throw the authorities off his trail, merlin.moncure@rcsonline.com ("Merlin Moncure") transmitted: > Alex wrote: >> How do you create a temporary view that has only a small subset of the >> data from the DB init? (Links to docs are fine - I can read ;). My >> query isn't all that complex, and my number of records might be from >> 10 to 2k depending on how I implement it. > > Well, you can't. My point was that the traditional query/view > approach is often more appropriate for these cases. Actually, you can if you assume you can "temporarily materialize" that view. You take the initial query and materialize it into a temporary table which can then be used to browse "detail." Thus, suppose you've got a case where the selection criteria draw in 8000 objects/transactions, of which you only want to fit 20/page. It's ugly and slow to process the 15th page, and you essentially reprocess the whole set from scratch each time: select [details] from [big table] where [criteria] order by [something] offset 280 limit 20; Instead, you might start out by doing: select [key fields] into temp table my_query from [big table] where [criteria]; create index my_query_idx on my_query(interesting fields); With 8000 records, the number of pages in the table will correspond roughly to the number of bytes per record which is probably pretty small. Then, you use a join on my_query to pull the bits you want: select [big table.details] from [big table], [select * from my_query order by [something] offset 280 limit 20] where [join criteria between my_query and big table] order by [something]; For this to be fast is predicated on my_query being compact, but that should surely be so. The big table may be 20 million records; for the result set to be even vaguely browsable means that my_query ought to be relatively small so you can pull subsets reasonably efficiently. This actually has a merit over looking at a dynamic, possibly-changing big table that you won't unexpectedly see the result set changing size. This strikes me as a pretty slick way to handle "data warehouse-style" browsing... -- output = ("cbbrowne" "@" "gmail.com") http://www.ntlug.org/~cbbrowne/oses.html The first cup of coffee recapitulates phylogeny.