From: Scott Marlowe <smarlowe@g2switchworks.com>
To: Brian Hurt <bhurt@janestcapital.com>
Cc: pgsql-performance@postgresql.org
Cc: pgsql-advocacy@postgresql.org
Subject: Re: [PERFORM] Postgres and really huge tables
Date: Thu, 18 Jan 2007 15:04:07 -0600
Message-ID: <1169154247.9586.100.camel@state.g2switchworks.com> (raw)
In-Reply-To: <45AFD927.9050605@janestcapital.com>
References: <4939.1164740825@sss.pgh.pa.us>
<200611282344.kASNijt09203@momjian.us>
<456CE066.50803@cheapcomplexdevices.com>
<456D8A65.3080400@janestcapital.com>
<1164812624.9690.500.camel@archimedes>
<456DA476.4000606@janestcapital.com>
<456DB8C9.7020206@cheapcomplexdevices.com>
<45AFD927.9050605@janestcapital.com>
On Thu, 2007-01-18 at 14:31, Brian Hurt wrote:
> Is there any experience with Postgresql and really huge tables? I'm
> talking about terabytes (plural) here in a single table. Obviously the
> table will be partitioned, and probably spread among several different
> file systems. Any other tricks I should know about?
>
> We have a problem of that form here. When I asked why postgres wasn't
> being used, the opinion that postgres would "just <explicitive> die" was
> given. Personally, I'd bet money postgres could handle the problem (and
> better than the ad-hoc solution we're currently using). But I'd like a
> couple of replies of the form "yeah, we do that here- no problem" to
> wave around.
It really depends on what you're doing.
Are you updating every row by a single user every hour, or are you
updating dozens of rows by hundreds of users at the same time?
PostgreSQL probably wouldn't die, but it may well be that for certain
batch processing operations it's a poorer choice than awk/sed or perl.
If you do want to tackle it with PostgreSQL, you'll likely want to build
a truly fast drive subsystem. Something like dozens to hundreds of
drives in a RAID-10 setup with battery backed cache, and a main server
with lots of memory on board.
But, really, it depends on what you're doing to the data.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: smarlowe@g2switchworks.com, bhurt@janestcapital.com, pgsql-advocacy@postgresql.org
Subject: Re: [PERFORM] Postgres and really huge tables
In-Reply-To: <1169154247.9586.100.camel@state.g2switchworks.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox