From: Simon Riggs <simon@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Stephen Frost <sfrost@snowman.net>
Cc: daveg <daveg@sonic.net>
Cc: pgsql-patches@postgresql.org
Subject: Re: pg_dump additional options for performance
Date: Mon, 21 Jul 2008 07:17:20 +0100
Message-ID: <1216621040.19656.867.camel@ebony.2ndQuadrant> (raw)
In-Reply-To: <18415.1216611275@sss.pgh.pa.us>
References: <20080720030749.GA18846@tamriel.snowman.net>
<1216529229.19656.835.camel@ebony.2ndQuadrant>
<1216535862.19656.845.camel@ebony.2ndQuadrant>
<20080720214335.GB18846@tamriel.snowman.net>
<1216591180.19656.848.camel@ebony.2ndQuadrant>
<20080721011829.GC18846@tamriel.snowman.net>
<20080721014942.GC30869@sonic.net>
<20080721015850.GD18846@tamriel.snowman.net>
<18415.1216611275@sss.pgh.pa.us>
On Sun, 2008-07-20 at 23:34 -0400, Tom Lane wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > * daveg (daveg@sonic.net) wrote:
> >> One observation, indexes should be built right after the table data
> >> is loaded for each table, this way, the index build gets a hot cache
> >> for the table data instead of having to re-read it later as we do now.
>
> > That's not how pg_dump has traditionally worked, and the point of this
> > patch is to add options to easily segregate the main pieces of the
> > existing pg_dump output (main schema definition, data dump, key/index
> > building). You suggestion brings up an interesting point that should
> > pg_dump's traditional output structure change the "--schema-post-load"
> > set of objects wouldn't be as clear to newcomers since the load and the
> > indexes would be interleaved in the regular output.
Stephen: Agreed.
> Yeah. Also, that is pushing into an entirely different line of
> development, which is to enable multithreaded pg_restore. The patch
> at hand is necessarily incompatible with that type of operation, and
> wouldn't be used together with it.
>
> As far as the documentation/definition aspect goes, I think it should
> just say the parts are
> * stuff needed before you can load the data
> * the data
> * stuff needed after loading the data
> and not try to be any more specific than that. There are corner cases
> that will turn any simple breakdown into a lie, and I doubt that it's
> worth trying to explain them all. (Take a close look at the dependency
> loop breaking logic in pg_dump if you doubt this.)
Tom: Agreed.
> I hadn't realized that Simon was using "pre-schema" and "post-schema"
> to name the first and third parts. I'd agree that this is confusing
> nomenclature: it looks like it's trying to say that the data is the
> schema, and the schema is not! How about "pre-data and "post-data"?
OK by me. Any other takers?
I also suggested having three options
--want-pre-schema
--want-data
--want-post-schema
so we could ask for any or all parts in the one dump. --data-only and
--schema-only are negative options so don't allow this.
(I don't like those names either, just thinking about capabilities)
--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-patches@postgresql.org
Cc: simon@2ndquadrant.com, tgl@sss.pgh.pa.us, sfrost@snowman.net, daveg@sonic.net
Subject: Re: pg_dump additional options for performance
In-Reply-To: <1216621040.19656.867.camel@ebony.2ndQuadrant>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox