From: Jenny Zhang <jenny@osdl.org>
To: Greg Stark <gsstark@mit.edu>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: perf-pgsql <pgsql-performance@postgresql.org>
Subject: Re: osdl-dbt3 run results - puzzled by the execution
Date: Fri, 19 Sep 2003 16:26:54 -0700
Message-ID: <1064014014.442.189.camel@ibm-a.pdx.osdl.net> (raw)
In-Reply-To: <87fzis2a84.fsf@stark.dyndns.tv>
References: <1063924610.31150.75.camel@ibm-a.pdx.osdl.net>
<8224.1063941637@sss.pgh.pa.us>
<87fzis2a84.fsf@stark.dyndns.tv>
On Fri, 2003-09-19 at 06:12, Greg Stark wrote:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>
> > I think this is a pipe dream. Variation in where the data gets laid
> > down on your disk drive would alone create more than that kind of delta.
> > I'm frankly amazed you could get repeatability within 2-3%.
>
> I think the reason he gets good repeatability is because he's talking about
> the aggregate results for a whole test run. Not individual queries. In theory
> you could just run the whole test multiple times. The more times you run it
> the lower the variation in the total run time would be.
>
That is right. The repeatability is due to the aggregate results for a
whole test run. As for individual query, the power test(single stream)
is very consistent, and the throughput test(multiple streams), any given
query execution time varies up to 15% if no swapping. If we set
sort_mem too high and swapping occurs, the variation is bigger.
> Actually, the variation in run time is also a useful statistic, both for
> postgres and the kernel. It might be useful to do multiple complete runs and
> keep track of the average standard deviation of the time required for each
> step.
>
I created a page with the execution time(in seconds), average, and
stddev for each query and each steps. The data is collected from 6 dbt3
runs.
http://developer.osdl.org/~jenny/pgsql-optimizer/exetime.html> Higher standard deviation implies queries can't be reliably depended on not to
> take inordinately long, which can be a problem for some working models. For
> the kernel it could mean latency issues or it could mean the swapper or buffer
> cache was overly aggressive.
I agree. I can think of another reason why the performance varies even
the swapper and buffer cache is not overly aggressive. Since PG depends
on OS to manage the buffer cache(correct me if I am wrong), it is up to
OS to decide what to keep in the cache. And OS can not anticipate what
is likely needed next.
Thanks,
Jenny
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: jenny@osdl.org, gsstark@mit.edu, tgl@sss.pgh.pa.us
Subject: Re: osdl-dbt3 run results - puzzled by the execution
In-Reply-To: <1064014014.442.189.camel@ibm-a.pdx.osdl.net>
References: <1063924610.31150.75.camel@ibm-a.pdx.osdl.net> <8224.1063941637@sss.pgh.pa.us> <87fzis2a84.fsf@stark.dyndns.tv> <1064014014.442.189.camel@ibm-a.pdx.osdl.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox