From: Luke Lonergan <llonergan@greenplum.com>
To: Bucky Jordan <bjordan@lumeta.com>
To: Spiegelberg, Greg <gspiegelberg@cranel.com>
To: Joshua Drake <jd@commandprompt.com>
Cc: Craig A. James <cjames@modgraph-usa.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Large tables (was: RAID 0 not as fast as expected)
Date: Mon, 18 Sep 2006 18:10:13 -0700
Message-ID: <C1349185.316F4%llonergan@greenplum.com> (raw)
In-Reply-To: <78ED28FACE63744386D68D8A9D1CF5D42099C7@MAIL.corp.lumeta.com>
Bucky,
On 9/18/06 7:37 AM, "Bucky Jordan" <bjordan@lumeta.com> wrote:
> My question is at what point do I have to get fancy with those big
> tables? From your presentation, it looks like PG can handle 1.2 billion
> records or so as long as you write intelligent queries. (And normal PG
> should be able to handle that, correct?)
PG has limitations that will confront you at sizes beyond about a couple
hundred GB of table size, as will Oracle and others.
You should be careful to implement very good disk hardware and leverage
Postgres 8.1 partitioning and indexes intelligently as you go beyond 100GB
per instance. Also be sure to set the random_page_cost parameter in
postgresql.conf to 100 or even higher when you use indexes, as the actual
seek rate for random access ranges between 50 and 300 for modern disk
hardware. If this parameter is left at the default of 4, indexes will often
be used inappropriately.
> Also, does anyone know if/when any of the MPP stuff will be ported to
> Postgres, or is the plan to keep that separate?
The plan is to keep that separate for now, though we're contributing
technology like partitioning, faster sorting, bitmap index, adaptive nested
loop, and hybrid hash aggregation to make big databases work better in
Postgres.
- Luke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: llonergan@greenplum.com, bjordan@lumeta.com, gspiegelberg@cranel.com, jd@commandprompt.com, cjames@modgraph-usa.com
Subject: Re: Large tables (was: RAID 0 not as fast as expected)
In-Reply-To: <C1349185.316F4%llonergan@greenplum.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox