X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 24052DBE8A for ; Sun, 27 Nov 2005 20:07:32 -0400 (AST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 54079-05 for ; Sun, 27 Nov 2005 20:07:33 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey- Received: from megazone.bigpanda.com (megazone.bigpanda.com [64.147.171.210]) by svr1.postgresql.org (Postfix) with ESMTP id 7F68EDBE5D for ; Sun, 27 Nov 2005 20:07:28 -0400 (AST) Received: by megazone.bigpanda.com (Postfix, from userid 1001) id 1E0DF3514F; Sun, 27 Nov 2005 16:07:32 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by megazone.bigpanda.com (Postfix) with ESMTP id 1C6083513F; Sun, 27 Nov 2005 16:07:32 -0800 (PST) Date: Sun, 27 Nov 2005 16:07:32 -0800 (PST) From: Stephan Szabo To: Luke Lonergan Cc: Postgresql Performance , David Lang Subject: Re: Hardware/OS recommendations for large databases ( In-Reply-To: Message-ID: <20051127160216.V96828@megazone.bigpanda.com> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=0 required=5 tests=[none] X-Spam-Score: 0 X-Spam-Level: X-Archive-Number: 200511/535 X-Sequence-Number: 15792 On Sun, 27 Nov 2005, Luke Lonergan wrote: > Stephan, > > On 11/27/05 7:48 AM, "Stephan Szabo" wrote: > > > On Sun, 27 Nov 2005, Luke Lonergan wrote: > > > >> Has anyone done the math.on the original post? 5TB takes how long to > >> scan once? If you want to wait less than a couple of days just for a > >> seq scan, you'd better be in the multi-gb per second range. > > > > Err, I get about 31 megabytes/second to do 5TB in 170,000 seconds. I think > > perhaps you were exaggerating a bit or adding additional overhead not > > obvious from the above. ;) > > Thanks - the calculator on my blackberry was broken ;-) Well, it was suspiciously close to a factor of 60 off, which when working in time could have just been a simple math error. > > At 1 gigabyte per second, 1 terrabyte should take about 1000 seconds > > (between 16 and 17 minutes). The impressive 3.2 gigabytes per second > > listed before (if it actually scans consistently at that rate), puts it at > > a little over 5 minutes I believe for 1, so about 26 for 5 terrabytes. > > The 200 megabyte per second number puts it about 7 hours for 5 > > terrabytes AFAICS. > > 7 hours, days, same thing ;-) > > On the reality of sustained scan rates like that: Well, the reason I asked was that IIRC the 3.2 used earlier in the discussion was exactly multiplying scanners and base rate (ie, no additional overhead). I couldn't tell if that was back of the envelope or if the overhead was in fact negligible. (Or I could be misremembering the conversation). I don't doubt that it's possible to get the rate, just wasn't sure if the rate was actually applicable to the ongoing discussion of the comparison.