From: Stephan Szabo <sszabo@megazone.bigpanda.com>
To: Luke Lonergan <llonergan@greenplum.com>
Cc: Postgresql Performance <pgsql-performance@postgresql.org>
Cc: David Lang <dlang@invendra.net>
Subject: Re: Hardware/OS recommendations for large databases (
Date: Sun, 27 Nov 2005 16:07:32 -0800 (PST)
Message-ID: <20051127160216.V96828@megazone.bigpanda.com> (raw)
In-Reply-To: <BFAF498C.14869%llonergan@greenplum.com>
References: <BFAF498C.14869%llonergan@greenplum.com>
On Sun, 27 Nov 2005, Luke Lonergan wrote:
> Stephan,
>
> On 11/27/05 7:48 AM, "Stephan Szabo" <sszabo@megazone.bigpanda.com> wrote:
>
> > On Sun, 27 Nov 2005, Luke Lonergan wrote:
> >
> >> Has anyone done the math.on the original post? 5TB takes how long to
> >> scan once? If you want to wait less than a couple of days just for a
> >> seq scan, you'd better be in the multi-gb per second range.
> >
> > Err, I get about 31 megabytes/second to do 5TB in 170,000 seconds. I think
> > perhaps you were exaggerating a bit or adding additional overhead not
> > obvious from the above. ;)
>
> Thanks - the calculator on my blackberry was broken ;-)
Well, it was suspiciously close to a factor of 60 off, which when working
in time could have just been a simple math error.
> > At 1 gigabyte per second, 1 terrabyte should take about 1000 seconds
> > (between 16 and 17 minutes). The impressive 3.2 gigabytes per second
> > listed before (if it actually scans consistently at that rate), puts it at
> > a little over 5 minutes I believe for 1, so about 26 for 5 terrabytes.
> > The 200 megabyte per second number puts it about 7 hours for 5
> > terrabytes AFAICS.
>
> 7 hours, days, same thing ;-)
>
> On the reality of sustained scan rates like that:
Well, the reason I asked was that IIRC the 3.2 used earlier in the
discussion was exactly multiplying scanners and base rate (ie, no
additional overhead). I couldn't tell if that was back of the envelope or
if the overhead was in fact negligible. (Or I could be misremembering the
conversation). I don't doubt that it's possible to get the rate, just
wasn't sure if the rate was actually applicable to the ongoing discussion
of the comparison.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: sszabo@megazone.bigpanda.com, llonergan@greenplum.com, dlang@invendra.net
Subject: Re: Hardware/OS recommendations for large databases (
In-Reply-To: <20051127160216.V96828@megazone.bigpanda.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox