From: Shaun Thomas <sthomas@optionshouse.com>
To: pgsql-performance@postgresql.org <pgsql-performance@postgresql.org>
Subject: Re: Performance of CLUSTER
Date: Mon, 11 Jun 2012 09:44:23 -0500
Message-ID: <4FD60447.1090802@optionshouse.com> (raw)
In-Reply-To: <4FD5FFEE.6000306@optrak.com>
References: <4FD458CA.3070902@optrak.com>
<4FD5F5DC.3080507@optionshouse.com>
<4FD5F6B5.2090505@optrak.com>
<4FD5F837.5080006@optionshouse.com>
<4FD5FA6F.3080903@optrak.com>
<4FD5FD5C.5040203@optionshouse.com>
<4FD5FFEE.6000306@optrak.com>
On 06/11/2012 09:25 AM, Mark Thornton wrote:
> Certainly not --- the server only has 5GB of memory. Nevertheless I
> don't expect quadratic behaviour for CLUSTER (n log n would be my
> expected time).
And there it is. :)
Since that's the case, *DO NOT* create the symlink from pgsql_tmp to
/dev/shm like I suggested before. You don't have enough memory for that,
and it will likely cause problems. I need to stop assuming everyone has
huge servers. I know low-end laptops have 4GB of RAM these days, but
servers have longer shelf-lives, and VMs can play larger roles.
So here's the thing, and I should have honestly realized it the second I
noted the >100x jump in execution time. All of your previous tables fit
in memory. Nice, speedy, >100x faster than disk, memory. It's not that
the table is only 10x larger than other tables in your examples, it's
that the entire thing doesn't fit in memory.
Since it can't just read the table and assume it's in memory, reads have
a chance to fetch from disk. Since it's also maintaining several
temporary files for the new index and replacement table structures, it's
fighting for random reads and writes during the whole process. That's in
addition to any transaction log traffic and checkpoints since the
process will span several.
Actually, your case is a good illustration of how memory and
high-performance IO devices can reduce maintenance costs. If you played
around with steadily increasing table sizes, I bet you could even find
the exact row count and table size where the table no longer fits in
PostgreSQL or OS cache, and suddenly takes 100x longer to process. That
kind of steady table growth is often seen in databases, and admins
sometimes see this without understanding why it happens.
--
Shaun Thomas
OptionsHouse | 141 W. Jackson Blvd. | Suite 500 | Chicago IL, 60604
312-444-8534
sthomas@optionshouse.com
______________________________________________
See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: sthomas@optionshouse.com
Subject: Re: Performance of CLUSTER
In-Reply-To: <4FD60447.1090802@optionshouse.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox