From: Kevin Grittner <Kevin.Grittner@wicourts.gov>
To: Shivakumar Ramannavar <shivasr@gmail.com>
To: pgsql-admin@postgresql.org
Subject: Re: My Query to insert and retrieve takes time
Date: Mon, 31 Oct 2011 10:33:33 -0500
Message-ID: <4EAE797D0200002500042899@gw.wicourts.gov> (raw)
In-Reply-To: <CACdOLXa12h6oN406mqZAZaCJG3G6hK=eTx_YhthJZU6vomPkAA@mail.gmail.com>
References: <CACdOLXa12h6oN406mqZAZaCJG3G6hK=eTx_YhthJZU6vomPkAA@mail.gmail.com>
Shivakumar Ramannavar <shivasr@gmail.com> wrote:
> my problem is that it takes 8.314 milliseconds to update one
> record
> EXPLAIN ANALYZE UPDATE terminal
> Update ... actual time=0.074..0.074
If you look at the EXPLAIN ANALYZE of the UPDATE, you'll see that
the UPDATE itself actually took only a small fraction of one ms.
My guess is that your write-ahead log (WAL) is on a 7200 or 7500 RPM
drive, and that you're committing each update separately. Further,
I would bet that you're not going through a RAID controller with
battery-backup (BBU) cache configured for write-back. And you're
also using a single connection to do all the updates.
Each commit must wait until the WAL is persisted, which can mean
waiting for a disk drive to spin all the way around again -- which
for a 7200 RPM drive takes 8.3 ms and for a 7500 RPM drive takes 8
ms. Without better hardware, you face a hard limit on how many
database transactions can commit on a single connection based on
that rotational delay.
> There are around 300,000 update operations and it is taking approx
> 80 min,
One way to solve the problem would be to BEGIN a transaction, do
your 300,000 updates, and then COMMIT the transaction. Another
would be to use a good RAID controller. A third would be to turn
off synchronized_commit. (Be sure to read the description of that
in the documentation to understand the implications.) Or you could
use a number of connections working in parallel.
-Kevin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: Kevin.Grittner@wicourts.gov, shivasr@gmail.com
Subject: Re: My Query to insert and retrieve takes time
In-Reply-To: <4EAE797D0200002500042899@gw.wicourts.gov>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox