From: Pierre Frédéric Caillaud <lists@peufeu.com>
To: Carlo Stonebanks <stonec.register@sympatico.ca>
To: pgsql-performance@postgresql.org
Subject: Re: Massive table (500M rows) update nightmare
Date: Sat, 09 Jan 2010 09:15:11 +0100
Message-ID: <op.u59ajlu4cke6l8@soyouz> (raw)
In-Reply-To: <hi7okp$2kca$1@news.hub.org>
References: <hi41os$eiv$1@news.hub.org>
<b1b9fac61001071305vf182f3ajff6827f92c943c68@mail.gmail.com>
<hi5kpr$184c$1@news.hub.org>
<dcc563d11001071740q572cdae2re410788fe790dbf6@mail.gmail.com>
<hi6ifm$1bi8$1@news.hub.org>
<dcc563d11001072221g66d3db9cr3556d49b2bdee47e@mail.gmail.com>
<hi7okp$2kca$1@news.hub.org>
>> crank it up more and delay the checkpoints as much as possible during
>> these updates. 64 segments is already 1024M.
>
> We have 425M rows, total table size is 78GB, so we can imagine a worst
> case UPDATE write is less than 200 bytes * number of rows specified in
> the update (is that logic correct?).
There is also the WAL : all these updates need to be logged, which doubles
the UPDATE write throughput. Perhaps you're WAL-bound (every 16MB segment
needs fsyncing), and tuning of fsync= and wal_buffers, or a faster WAL
disk could help ? (I don't remember your config).
> Inerestingly, the total index size is 148GB, twice that of the table,
> which may be an indication of where the performance bottleneck is.
Index updates can create random I/O (suppose you have a btree on a rather
random column)...
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: lists@peufeu.com, stonec.register@sympatico.ca
Subject: Re: Massive table (500M rows) update nightmare
In-Reply-To: <op.u59ajlu4cke6l8@soyouz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox