From: Steven Flatt <steven.flatt@gmail.com>
To: pgsql-performance@postgresql.org
Subject: Insertion to temp table deteriorating over time
Date: Wed, 13 Dec 2006 11:44:19 -0500
Message-ID: <357fa7590612130844q58ef3ca5q31e164f9b006e0fc@mail.gmail.com> (raw)
Hi,
Our application is using Postgres 7.4 and I'd like to understand the root
cause of this problem:
To speed up overall insert time, our application will write thousands of
rows, one by one, into a temp table (same structure as a permanent table),
then do a bulk insert from the temp table to the permanent table. After
this bulk insert is done, the temp table is truncated and the process is
repeated. We do this because Postgres can do many individual inserts to a
temp table much faster than to a permanent table.
The problem we are seeing is that over time, the cost of a single insert to
the temp table seems to grow. After a restart of postgres, a single insert
to the temp table takes about 3ms. Over a few days, this grows to about
60ms per insert. Restarting postgres drops this insert time back to 3ms,
supposedly because the temp table is re-created. Our workaround right now
is to restart the database every few days, but we don't like this solution
much.
Any idea where the bloat is happening? I believe that if we were dropping
and re-creating the temp table over and over, that could result in pg_class
bloat (among other catalog tables), but what is going wrong if we use the
same table over and over and truncate it?
Thanks,
Steve
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: steven.flatt@gmail.com
Subject: Re: Insertion to temp table deteriorating over time
In-Reply-To: <357fa7590612130844q58ef3ca5q31e164f9b006e0fc@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox