X-Original-To: pgsql-performance@postgresql.org Received: from spampd.localdomain (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 5D2D24758E6 for ; Tue, 8 Apr 2003 19:22:45 -0400 (EDT) Received: from filer (12-234-86-219.client.attbi.com [12.234.86.219]) by postgresql.org (Postfix) with ESMTP id A29CE474E42 for ; Tue, 8 Apr 2003 19:22:44 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) (uid 1000) by filer with local; Tue, 08 Apr 2003 16:22:47 -0700 Date: Tue, 8 Apr 2003 16:22:47 -0700 From: Kevin Brown To: pgsql-performance@postgresql.org Cc: Josh Berkus Subject: Re: ext3 filesystem / linux 7.3 Message-ID: <20030408232247.GH1847@filer> Mail-Followup-To: Kevin Brown , pgsql-performance@postgresql.org, Josh Berkus References: <231C6A82-65C3-11D7-8D88-0003939CC61E@trilug.org> <200304030852.34194.josh@agliodbs.com> <0b7a01c2fa04$2d299bf0$0b02a8c0@pointhere.net> <200304030943.06406.josh@agliodbs.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline In-Reply-To: <200304030943.06406.josh@agliodbs.com> User-Agent: Mutt/1.4i Organization: Frobozzco International X-Spam-Status: No, hits=-38.0 required=5.0 tests=BAYES_10,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,REPLY_WITH_QUOTES,USER_AGENT_MUTT autolearn=ham version=2.50 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.50 (1.173-2003-02-20-exp) X-Archive-Number: 200304/76 X-Sequence-Number: 1582 Josh Berkus wrote: > Jeffery, > > > Can't we generate data? Random data stored in random formats at random > > sizes would stress the file system wouldn't it? > > In my experience, randomly generated data tends to resemble real data very > little in distribution patterns and data types. This is one of the > limitations of PGBench. Okay, from this it sounds like what we need is information on the data types typically used for real world applications and information on the the distribution patterns for each type (the latter could get quite complex and varied, I'm sure, but since we're after something that's typical, we only need a few examples). So perhaps the first step in this is to write something that will show what the distribution pattern for data in a table is? With that information, we *could* randomly generate data that would conform to the statistical patterns seen in the real world. In fact, even though the databases you have access to are all proprietary, I'm pretty sure their owners would agree to let you run a program that would gather statistical distribution about it. Then (as long as they agree) you could copy the schema itself, recreate it on the test system, and randomly generate the data. -- Kevin Brown kevin@sysexperts.com