X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 2573954F47 for ; Fri, 4 Mar 2005 19:56:30 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 70799-09 for ; Fri, 4 Mar 2005 19:56:28 +0000 (GMT) Received: from ct.radiology.uiowa.edu (ct.radiology.uiowa.edu [129.255.60.186]) by svr1.postgresql.org (Postfix) with ESMTP id 7C7A454E34 for ; Fri, 4 Mar 2005 19:56:27 +0000 (GMT) Received: from [192.168.1.11] (12-217-241-0.client.mchsi.com [12.217.241.0]) by ct.radiology.uiowa.edu (8.11.6/8.11.6) with ESMTP id j24JuJ310009; Fri, 4 Mar 2005 13:56:20 -0600 Message-ID: <4228BD5C.9020800@arbash-meinel.com> Date: Fri, 04 Mar 2005 13:56:12 -0600 From: John A Meinel User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ken Cc: Richard Huxton , pgsql-performance@postgresql.org Subject: Re: Help with tuning this query (with explain analyze finally) References: <001001c51ef4$3849eeb0$6a01a8c0@KEN> <200503022152.13211.josh@agliodbs.com> <004601c51fbe$828bbb50$6a01a8c0@KEN> <200503030935.14537.josh@agliodbs.com> <001d01c5204a$b440edf0$6a01a8c0@KEN> <4227AA36.2080700@arbash-meinel.com> <000e01c5207a$1f668ac0$6a01a8c0@KEN> <42288529.7000803@archonet.com> <000e01c520d8$4f899a90$780ba8c0@javadude> <42289347.7040607@arbash-meinel.com> In-Reply-To: <42289347.7040607@arbash-meinel.com> X-Enigmail-Version: 0.89.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF3BC37F711E5725CDBD5F520" X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests= X-Spam-Level: X-Archive-Number: 200503/80 X-Sequence-Number: 10785 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF3BC37F711E5725CDBD5F520 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit John Arbash Meinel wrote: > Ken wrote: > >> Richard, >> >> What do you mean by summary table? Basically a cache of the query >> into a table with replicated column names of all the joins? I'd >> probably have to whipe out the table every minute and re-insert the >> data for each carrier in the system. I'm not sure how expensive this >> operation would be, but I'm guessing it would be fairly heavy-weight. >> And maintaince would be a lot harder because of the duplicated >> columns, making refactorings on the database more error-prone. Am I >> understanding your suggestion correctly? Please correct me if I am. >> >>> Can you turn the problem around? Calculate what you want for all >>> users (once every 60 seconds) and stuff those results into a summary >>> table. Then let the users query the summary table as often as they >>> like (with the understanding that the figures aren't going to update >>> any faster than once a minute) >> >> > It's the same idea of a materialized view, or possibly just a lazy cache. > ... > This unrolls all of the work, a table which should be really fast to > query. If this query takes less than 10s to generate, than just have a > service run it every 60s. I think for refreshing, it is actually faster > to drop the table and recreate it, rather than deleteing the entries. > Dropping also has the advantage that if you ever add more rows to s or > ss, then the table automatically gets the new entries. > Just as a small update. If completely regenerating the cache takes to long, the other way to do it, is to create insert and update triggers on s and ss, such that as they change, they also update the cachedview table. Something like CREATE TRIGGER on_ss_ins AFTER INSERT ON ss FOR EACH ROW EXECUTE INSERT INTO cached_view SELECT p.id as person_id, s.*, ss.* FROM WHERE s.id = NEW.id; This runs the same query, but notice that the WHERE means it only allows the new row. So this query should run fast. It is a little bit of overhead on each of your inserts, but it should keep the cache up-to-date. With something like this, I would have the final client query still include the date restriction, since you accumulate older rows into the cached view. But you can run a daily process that prunes out everything older than 31 days, which keeps the cachedview from getting really large. John =:-> --------------enigF3BC37F711E5725CDBD5F520 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (Cygwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCKL1fJdeBCYSNAAMRAoQ2AKCrcTSv3Kc21WdAzFq5LsJx1RbBdACgw78C IQWDOtHPh4MBTEVTpYmrCEA= =zjI/ -----END PGP SIGNATURE----- --------------enigF3BC37F711E5725CDBD5F520--