From: fork <forkandwait@gmail.com>
To: pgsql-performance@postgresql.org
Subject: Tuning massive UPDATES and GROUP BY's?
Date: Thu, 10 Mar 2011 15:40:53 +0000 (UTC)
Message-ID: <loom.20110310T163020-437@post.gmane.org> (raw)
Given that doing a massive UPDATE SET foo = bar || ' ' || baz; on a 12 million
row table (with about 100 columns -- the US Census PUMS for the 2005-2009 ACS)
is never going to be that fast, what should one do to make it faster?
I set work_mem to 2048MB, but it currently is only using a little bit of memory
and CPU. (3% and 15% according to top; on a SELECT DISTINCT ... LIMIT earlier,
it was using 70% of the memory).
The data is not particularly sensitive; if something happened and it rolled
back, that wouldnt be the end of the world. So I don't know if I can use
"dangerous" setting for WAL checkpoints etc. There are also aren't a lot of
concurrent hits on the DB, though a few.
I am loathe to create a new table from a select, since the indexes themselves
take a really long time to build.
As the title alludes, I will also be doing GROUP BY's on the data, and would
love to speed these up, mostly just for my own impatience...
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: forkandwait@gmail.com
Subject: Re: Tuning massive UPDATES and GROUP BY's?
In-Reply-To: <loom.20110310T163020-437@post.gmane.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox