From: Markus Schaber <schabi@logix-tt.com>
To: Worky Workerson <worky.workerson@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Best COPY Performance
Date: Mon, 23 Oct 2006 11:27:41 +0200
Message-ID: <453C8B0D.4070803@logix-tt.com> (raw)
In-Reply-To: <ce4072df0610201305u7836dc7bn437ec0aaf1c3e25f@mail.gmail.com>
References: <ce4072df0610201305u7836dc7bn437ec0aaf1c3e25f@mail.gmail.com>
Hi, Worky,
Worky Workerson wrote:
> I am currently getting
> between 10K and 15K inserts/second.
> I ran trivial little insert into a table with a single integer row and
> came close to 250K inserts/second using psql's \copy, so I'm thinking
> that my code could be optimized a bit more, but wanted to check around
> to see if that was the case.
Could you COPY one of your tables out to disk via psql, and then COPY it
back into the database, to reproduce this measurement with your real data?
Also, how much is the disk load, and CPU usage?
As long as psql is factor 20 better than your perl script, I think that
the perl interface is what should be optimized.
On a table with no indices, triggers and contstraints, we managed to
COPY about 7-8 megabytes/second with psql over our 100 MBit network, so
here the network was the bottleneck.
You should think about making your perl program writing the COPY
statement as text, and piping it into psql.
HTH,
Markus
--
Markus Schaber | Logical Tracking&Tracing International AG
Dipl. Inf. | Software Development GIS
Fight against software patents in Europe! www.ffii.org
www.nosoftwarepatents.org
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: schabi@logix-tt.com, worky.workerson@gmail.com
Subject: Re: Best COPY Performance
In-Reply-To: <453C8B0D.4070803@logix-tt.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox