From: Luke Lonergan <llonergan@greenplum.com>
To: Mark Kirkwood <markir@paradise.net.nz>
Cc: Dave Cramer <pg@fastcrypt.com>
Cc: Greg Stark <gsstark@mit.edu>
Cc: Joshua Marsh <icub3d@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Hardware/OS recommendations for large databases (
Date: Fri, 18 Nov 2005 16:04:00 -0800
Message-ID: <BFA3ABF0.14124%llonergan@greenplum.com> (raw)
In-Reply-To: <437E67EE.4070605@paradise.net.nz>
Mark,
On 11/18/05 3:46 PM, "Mark Kirkwood" <markir@paradise.net.nz> wrote:
> If you alter this to involve more complex joins (e.g 4. way star) and
> (maybe add a small number of concurrent executors too) - is it still the
> case?
4-way star, same result, that's part of my point. With Bizgres MPP, the
4-way star uses 4 concurrent scanners, though not all are active all the
time. And that's per segment instance - we normally use one segment
instance per CPU, so our concurrency is NCPUs plus some.
The trick is the "small number of concurrent executors" part. The only way
to get this with normal postgres is to have concurrent users, and normally
they are doing different things, scanning different parts of the disk.
These are competing things, and for concurrency enhancement something like
"sync scan" would be an effective optimization.
But in reporting, business analytics and warehousing in general, there are
reports that take hours to run. If you can knock that down by factors of 10
using parallelism, it's a big win. That's the reason that Teradata did $1.5
Billion in business last year.
More importantly - that's the kind of work that everyone using internet data
for analytics wants right now.
- Luke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: llonergan@greenplum.com, markir@paradise.net.nz, pg@fastcrypt.com, gsstark@mit.edu, icub3d@gmail.com
Subject: Re: Hardware/OS recommendations for large databases (
In-Reply-To: <BFA3ABF0.14124%llonergan@greenplum.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox