From: Simon Riggs <simon@2ndquadrant.com>
To: Dimitri Fontaine <dfontaine@hi-media.com>
Cc: pgsql-general@postgresql.org
Subject: Re: [PERFORM] multi-threaded pgloader makes it in version 2.3.0
Date: Mon, 10 Mar 2008 17:14:23 +0000
Message-ID: <1205169263.4269.169.camel@ebony.site> (raw)
In-Reply-To: <200803101718.16866.dfontaine@hi-media.com>
References: <200802261308.53459.dfontaine@hi-media.com>
<1204395453.8258.117.camel@ebony.site>
<200803101718.16866.dfontaine@hi-media.com>
On Mon, 2008-03-10 at 17:18 +0100, Dimitri Fontaine wrote:
> Le samedi 01 mars 2008, Simon Riggs a écrit :
> > On Tue, 2008-02-26 at 13:08 +0100, Dimitri Fontaine wrote:
> > > I'd like to have some feedback about the new version, in term of bugs
> > > encountered and performance limitations (is pgloader up to what you would
> > > expect a multi-threaded loader to be at?)
> >
> > Maybe post to general as well if you don't get any replies here.
> > New feature is very important for us.
>
> So, here's yet another mail about pgloader new 2.3.0 version, please forgive
> me for being over zealous here if that's how I appear to be to you...
>
> Those links will give you detailed information about the new release.
> http://pgfoundry.org/projects/pgloader
> http://pgfoundry.org/forum/forum.php?forum_id=1283
> http://pgloader.projects.postgresql.org/#_parallel_loading
Sounds good.
Not sure when or why I would want an rrqueue_size larger than
copy_every, and less sounds very strange. Can we get away with it being
the same thing in all cases?
Do you have some basic performance numbers? It would be good to
understand the overhead of the parallelism on a large file with 1, 2 and
4 threads. Would be good to see if synchronous_commit = off helped speed
things up as well.
Presumably -V and -T still work when we go parallel, but just issue one
query?
--
Simon Riggs
2ndQuadrant http://www.2ndQuadrant.com
PostgreSQL UK 2008 Conference: http://www.postgresql.org.uk
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: simon@2ndquadrant.com, dfontaine@hi-media.com, pgsql-general@postgresql.org
Subject: Re: [PERFORM] multi-threaded pgloader makes it in version 2.3.0
In-Reply-To: <1205169263.4269.169.camel@ebony.site>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox