From: John A Meinel <john@arbash-meinel.com>
To: Ken <ken@upfactor.com>
Cc: Richard Huxton <dev@archonet.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Help with tuning this query (with explain analyze finally)
Date: Fri, 04 Mar 2005 13:56:12 -0600
Message-ID: <4228BD5C.9020800@arbash-meinel.com> (raw)
In-Reply-To: <42289347.7040607@arbash-meinel.com>
References: <001001c51ef4$3849eeb0$6a01a8c0@KEN>
<200503022152.13211.josh@agliodbs.com>
<004601c51fbe$828bbb50$6a01a8c0@KEN>
<200503030935.14537.josh@agliodbs.com>
<001d01c5204a$b440edf0$6a01a8c0@KEN>
<4227AA36.2080700@arbash-meinel.com>
<000e01c5207a$1f668ac0$6a01a8c0@KEN>
<42288529.7000803@archonet.com>
<000e01c520d8$4f899a90$780ba8c0@javadude>
<42289347.7040607@arbash-meinel.com>
John Arbash Meinel wrote:
> Ken wrote:
>
>> Richard,
>>
>> What do you mean by summary table? Basically a cache of the query
>> into a table with replicated column names of all the joins? I'd
>> probably have to whipe out the table every minute and re-insert the
>> data for each carrier in the system. I'm not sure how expensive this
>> operation would be, but I'm guessing it would be fairly heavy-weight.
>> And maintaince would be a lot harder because of the duplicated
>> columns, making refactorings on the database more error-prone. Am I
>> understanding your suggestion correctly? Please correct me if I am.
>>
>>> Can you turn the problem around? Calculate what you want for all
>>> users (once every 60 seconds) and stuff those results into a summary
>>> table. Then let the users query the summary table as often as they
>>> like (with the understanding that the figures aren't going to update
>>> any faster than once a minute)
>>
>>
> It's the same idea of a materialized view, or possibly just a lazy cache.
>
...
> This unrolls all of the work, a table which should be really fast to
> query. If this query takes less than 10s to generate, than just have a
> service run it every 60s. I think for refreshing, it is actually faster
> to drop the table and recreate it, rather than deleteing the entries.
> Dropping also has the advantage that if you ever add more rows to s or
> ss, then the table automatically gets the new entries.
>
Just as a small update. If completely regenerating the cache takes to
long, the other way to do it, is to create insert and update triggers on
s and ss, such that as they change, they also update the cachedview table.
Something like
CREATE TRIGGER on_ss_ins AFTER INSERT ON ss FOR EACH ROW EXECUTE
INSERT INTO cached_view SELECT p.id as person_id, s.*, ss.* FROM
<the big stuff> WHERE s.id = NEW.id;
This runs the same query, but notice that the WHERE means it only allows
the new row. So this query should run fast. It is a little bit of
overhead on each of your inserts, but it should keep the cache
up-to-date. With something like this, I would have the final client
query still include the date restriction, since you accumulate older
rows into the cached view. But you can run a daily process that prunes
out everything older than 31 days, which keeps the cachedview from
getting really large.
John
=:->
Attachments:
signature.asc (application/pgp-signature, 252B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: john@arbash-meinel.com, ken@upfactor.com, dev@archonet.com
Subject: Re: Help with tuning this query (with explain analyze finally)
In-Reply-To: <4228BD5C.9020800@arbash-meinel.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox