From: Merlin Moncure <mmoncure@gmail.com>
To: femski <hypertree@yahoo.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Postgres batch write very slow - what to do
Date: Thu, 15 Mar 2007 07:56:20 +0530
Message-ID: <b42b73150703141926p220b86c3m663f12aabd678d78@mail.gmail.com> (raw)
In-Reply-To: <9486986.post@talk.nabble.com>
References: <9452092.post@talk.nabble.com>
<45F685B7.5040001@enterprisedb.com>
<9453712.post@talk.nabble.com>
<9473692.post@talk.nabble.com>
<b42b73150703141752q64ddebc2mb53c501047d0b7b6@mail.gmail.com>
<9486986.post@talk.nabble.com>
On 3/15/07, femski <hypertree@yahoo.com> wrote:
>
> I am using Oracle XE so its using only one core and doing just fine.
> How do I split backend to 4+ processes ?
> I don't want to write a multithreaded loader app.
> And I didn't think Postgres can utilize multiple cores for the
> same insert statement.
well, what sql is the jdbc driver creating exactly? It is probably
running inserts in a transaction. your load is about 17k inserts/sec
which about right for postgres on your hardware. you have the
following options to play increase insert performance:
* tweak postgresql.conf
fsync: off it is not already
wal_segments: bump to at least 24 or so
maintenance_work_mem: if you create key after insert, bump this high
(it speeds create index)
bgwriter settings: you can play with these, try disabling bgwriter
first (maxpages=0)
full_page_writes=off might help, not 100% sure about this
* distribute load
make load app multi thrreaded.
* use copy for bulk load
[is there a way to make jdbc driver do this?]
* use multi-line inserts (at least 10 rows/insert)...nearly as fast as copy
* if jdbc driver is not already doing so, prepare your statements and execute.
merlin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: mmoncure@gmail.com, hypertree@yahoo.com
Subject: Re: Postgres batch write very slow - what to do
In-Reply-To: <b42b73150703141926p220b86c3m663f12aabd678d78@mail.gmail.com>
References: <9452092.post@talk.nabble.com> <45F685B7.5040001@enterprisedb.com> <9453712.post@talk.nabble.com> <9473692.post@talk.nabble.com> <b42b73150703141752q64ddebc2mb53c501047d0b7b6@mail.gmail.com> <9486986.post@talk.nabble.com> <b42b73150703141926p220b86c3m663f12aabd678d78@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox