From: femski <hypertree@yahoo.com>
To: pgsql-performance@postgresql.org
Subject: Re: Postgres batch write very slow - what to do
Date: Thu, 15 Mar 2007 04:52:53 -0700 (PDT)
Message-ID: <9492938.post@talk.nabble.com> (raw)
In-Reply-To: <b42b73150703141926p220b86c3m663f12aabd678d78@mail.gmail.com>
References: <9452092.post@talk.nabble.com>
<45F685B7.5040001@enterprisedb.com>
<9453712.post@talk.nabble.com>
<9473692.post@talk.nabble.com>
<b42b73150703141752q64ddebc2mb53c501047d0b7b6@mail.gmail.com>
<9486986.post@talk.nabble.com>
<b42b73150703141926p220b86c3m663f12aabd678d78@mail.gmail.com>
I tried maxpages = 0 and full_page_writes=off and it seemed to be taking
forever.
All other tricks I have already tried.
At this point I wondering if its a jdbc client side issue - I am using the
latest 8.1.
(as I said in an earlier post - I am using addBatch with batch size of 100).
But just in case - I am missing something.
If 17k record/sec is right around expected then I must say I am little
disappointed from the "most advanced open source database".
thanks for all your help.
-Sanjay
Merlin Moncure-2 wrote:
>
> On 3/15/07, femski <hypertree@yahoo.com> wrote:
> well, what sql is the jdbc driver creating exactly? It is probably
> running inserts in a transaction. your load is about 17k inserts/sec
> which about right for postgres on your hardware. you have the
> following options to play increase insert performance:
>
> * tweak postgresql.conf
> fsync: off it is not already
> wal_segments: bump to at least 24 or so
> maintenance_work_mem: if you create key after insert, bump this high
> (it speeds create index)
> bgwriter settings: you can play with these, try disabling bgwriter
> first (maxpages=0)
> full_page_writes=off might help, not 100% sure about this
>
> * distribute load
> make load app multi thrreaded.
>
> * use copy for bulk load
> [is there a way to make jdbc driver do this?]
>
> * use multi-line inserts (at least 10 rows/insert)...nearly as fast as
> copy
>
> * if jdbc driver is not already doing so, prepare your statements and
> execute.
>
> merlin
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>
>
--
View this message in context: http://www.nabble.com/Postgres-batch-write-very-slow---what-to-do-tf3395195.html#a9492938
Sent from the PostgreSQL - performance mailing list archive at Nabble.com.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: hypertree@yahoo.com
Subject: Re: Postgres batch write very slow - what to do
In-Reply-To: <9492938.post@talk.nabble.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox