X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [64.117.224.130]) by svr1.postgresql.org (Postfix) with ESMTP id DBBA8D1B51D for ; Fri, 19 Sep 2003 23:19:25 +0000 (GMT) Received: from svr1.postgresql.org ([64.117.224.193]) by localhost (neptune.hub.org [64.117.224.130]) (amavisd-new, port 10024) with ESMTP id 90392-09 for ; Fri, 19 Sep 2003 20:19:18 -0300 (ADT) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by svr1.postgresql.org (Postfix) with ESMTP id 665D7D1B57C for ; Fri, 19 Sep 2003 20:19:13 -0300 (ADT) Received: from ibm-a.pdx.osdl.net (ibm-a.pdx.osdl.net [172.20.1.50]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id h8JNJ2U08433; Fri, 19 Sep 2003 16:19:02 -0700 Subject: Re: osdl-dbt3 run results - puzzled by the execution From: Jenny Zhang To: Greg Stark Cc: Tom Lane , perf-pgsql In-Reply-To: <87fzis2a84.fsf@stark.dyndns.tv> References: <1063924610.31150.75.camel@ibm-a.pdx.osdl.net> <8224.1063941637@sss.pgh.pa.us> <87fzis2a84.fsf@stark.dyndns.tv> Content-Type: text/plain Organization: osdl Message-Id: <1064014014.442.189.camel@ibm-a.pdx.osdl.net> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.4 Date: Fri, 19 Sep 2003 16:26:54 -0700 Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200309/288 X-Sequence-Number: 3654 On Fri, 2003-09-19 at 06:12, Greg Stark wrote: > Tom Lane writes: > > > I think this is a pipe dream. Variation in where the data gets laid > > down on your disk drive would alone create more than that kind of delta. > > I'm frankly amazed you could get repeatability within 2-3%. > > I think the reason he gets good repeatability is because he's talking about > the aggregate results for a whole test run. Not individual queries. In theory > you could just run the whole test multiple times. The more times you run it > the lower the variation in the total run time would be. > That is right. The repeatability is due to the aggregate results for a whole test run. As for individual query, the power test(single stream) is very consistent, and the throughput test(multiple streams), any given query execution time varies up to 15% if no swapping. If we set sort_mem too high and swapping occurs, the variation is bigger. > Actually, the variation in run time is also a useful statistic, both for > postgres and the kernel. It might be useful to do multiple complete runs and > keep track of the average standard deviation of the time required for each > step. > I created a page with the execution time(in seconds), average, and stddev for each query and each steps. The data is collected from 6 dbt3 runs. http://developer.osdl.org/~jenny/pgsql-optimizer/exetime.html > Higher standard deviation implies queries can't be reliably depended on not to > take inordinately long, which can be a problem for some working models. For > the kernel it could mean latency issues or it could mean the swapper or buffer > cache was overly aggressive. I agree. I can think of another reason why the performance varies even the swapper and buffer cache is not overly aggressive. Since PG depends on OS to manage the buffer cache(correct me if I am wrong), it is up to OS to decide what to keep in the cache. And OS can not anticipate what is likely needed next. Thanks, Jenny