From: Tom Lane <tgl@sss.pgh.pa.us>
To: Jon Nelson <jnelson+pgsql@jamponi.net>
Cc: pgsql-performance@postgresql.org
Subject: Re: external sort performance
Date: Thu, 17 Nov 2011 12:55:20 -0500
Message-ID: <27243.1321552520@sss.pgh.pa.us> (raw)
In-Reply-To: <CAKuK5J3taH85P6zc1j63mkwiNXH2nfJ5ER6R1+Vk5iG_Cq-cFw@mail.gmail.com>
References: <CAKuK5J3taH85P6zc1j63mkwiNXH2nfJ5ER6R1+Vk5iG_Cq-cFw@mail.gmail.com>
Jon Nelson <jnelson+pgsql@jamponi.net> writes:
> This is on PG 8.4.8 on Linux, 16GB of "real" RAM.
> Most recently, I enabled trace_sort, disabled hash aggregation[1], and
> set a large work_mem (normally very small, in this case I tried
> anything from 8MB to 256MB. I even tried 1GB and 2GB).
FWIW, I think hash aggregation is your best shot at getting reasonable
performance. Sorting 175GB of data is going to hurt no matter what.
If the grouped table amounts to 5GB, I wouldn't have expected the hash
table to be more than maybe 2-3X that size (although this does depend on
what aggregates you're running...). Letting the hash aggregation have
all your RAM might be the best answer.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: tgl@sss.pgh.pa.us, jnelson+pgsql@jamponi.net
Subject: Re: external sort performance
In-Reply-To: <27243.1321552520@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox