From: Thomas Finneid <tfinneid@student.matnat.uio.no>
Cc: pgsql-performance@postgresql.org
Subject: Re: improvement suggestions for performance design
Date: Sun, 08 Jul 2007 12:06:29 +0200
Message-ID: <4690B725.5070303@ifi.uio.no> (raw)
In-Reply-To: <c637d8bb0707060430x1656a796y728c88e9e2103ea5@mail.gmail.com>
References: <40362.134.32.140.234.1183637748.squirrel@webmail.uio.no>
<468CEB32.30109@enterprisedb.com>
<40891.134.32.140.234.1183642557.squirrel@webmail.uio.no>
<468D3AF7.4070505@enterprisedb.com>
<468D6F68.7090903@ifi.uio.no>
<c637d8bb0707060430x1656a796y728c88e9e2103ea5@mail.gmail.com>
Kalle Hallivuori wrote:
> COPY is plentitudes faster than INSERT:
> http://www.postgresql.org/docs/8.1/interactive/sql-copy.html
>
> If you can't just push the data straight into the final table with
> COPY, push it into a temporary table that you go through with the
> database procedure.
>
> Shameless plug: If you use Java and miss COPY functionality in the
> driver, it's available at
>
> http://kato.iki.fi/sw/db/postgresql/jdbc/copy/
>
> I was able to practically nullify time spent inserting with that.
Interresting, I will definately have a look at it.
What is the maturity level of the code at this point? and what is
potentially missing to bring it up to production quality? (stability is
of the utmost importance in my usage scenario.)
>
>> Well, it has been tested and showed to make postgres perform much
>> better, ie. 100 000 inserts separated between 4 threads performed much
>> faster than with a single thread alone.
>
> Sounds interesting. The results should still end up written into the
> same table, so are you sure this didn't end up using the same time at
> server end - would that even matter to you?
yes it would matter, because a number of clients are waiting to read the
data before the next batch of data is inserted. (in essence every 6
seconds 40000 attributes must be written, and after that write 8-16
clients read most of that data based on query criteria.and this is just
today, in the future, 5-10 years, it might be as high as 2-300 000
attributes per 3 seconds.
thomas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: tfinneid@student.matnat.uio.no
Subject: Re: improvement suggestions for performance design
In-Reply-To: <4690B725.5070303@ifi.uio.no>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox