Date: Sat, 26 Jul 2008 11:11:10 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Simon Riggs <simon@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-patches@postgresql.org
Subject: Re: pg_dump additional options for performance
Message-ID: <20080726151110.GF16005@tamriel.snowman.net>
Mail-Followup-To: Simon Riggs <simon@2ndquadrant.com>,
	Tom Lane <tgl@sss.pgh.pa.us>, pgsql-patches@postgresql.org
References: <1216621040.19656.867.camel@ebony.2ndQuadrant>
	<20080721115602.GG18846@tamriel.snowman.net>
	<1216831237.3894.676.camel@ebony.2ndQuadrant>
	<1216832987.3894.678.camel@ebony.2ndQuadrant>
	<20080724032047.GI18846@tamriel.snowman.net>
	<1216885031.3894.739.camel@ebony.2ndQuadrant>
	<21687.1216886041@sss.pgh.pa.us>
	<1216889809.3894.747.camel@ebony.2ndQuadrant>
	<2818.1217027784@sss.pgh.pa.us>
	<1217063104.3894.1100.camel@ebony.2ndQuadrant>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="yRA+Bmk8aPhU85Qt"
Content-Disposition: inline
In-Reply-To: <1217063104.3894.1100.camel@ebony.2ndQuadrant>
User-Agent: Mutt/1.5.18 (2008-05-17)


--yRA+Bmk8aPhU85Qt
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

* Simon Riggs (simon@2ndquadrant.com) wrote:
> The key capability here is being able to split the dump into multiple
> pieces. The equivalent capability on restore is *not* required, because
> once the dump has been split the restore never needs to be. It might
> seem that the patch should be symmetrical with respect to pg_dump and
> pg_restore, but I see no use case for the pg_restore case.

I'm inclined to agree with this.  It might have been nice to provide a
way to split out already-created dumps, but I suspect that people who
need that probably have already figured out a way to do it (I know I
have..).  We should probably ensure that pg_restore doesn't *break* when
fed a partial dump.

> > The patch as submitted enforces what seem largely arbitrary restrictions
> > on combining these switches.
>=20
> I had it both ways at various points in development. I'm happy with what
> you propose.

I agree with removing the restrictions.  I don't see much in the way of
use cases, but it reduces code and doesn't cause problems.

> > Another issue is that the rules for deciding which objects are "before
> > data" and which are "after data" are wrong.  In particular ACLs are aft=
er
> > data not before data, which is relatively easy to fix. =20
>=20
> OK

This was partially why I was complaining about having documentation, and
a policy for that matter, which goes into more detail about why X is before
or after the data.  I agree that they're after today, but I don't see
any particular reason why they should be one or the other.  If we
adopted a policy like I proposed (--schema-post-data is essentially that
which uses the data and is faster done in bulk) then ACLs would be
before, and I tend to feel like it makes more sense that way.

> > Not so easy to fix
> > is that COMMENTs might be either before or after data depending on what
> > kind of object they are attached to.
>=20
> Is there anything to fix? Comments are added by calls to dumpComment,
> which are always made in conjunction with the dump of an object. So if
> you dump the object you dump the comment. As long as objects are
> correctly split out then comments will be also.

I agree with this, and it follows for BLOB comments- in any case, they
go with the object being dumped at the time of that object getting
dumped.  Comments make sense as an extention of the object, not as a
seperate set of objects to be explicitly placed before or after the
data.

> All of the above makes me certain I want to remove these options from
> pg_restore.

I'm in agreement with this.

> > BTW, another incomplete item is that pg_dumpall should probably be taug=
ht
> > to accept and pass down --schema-before-data and --schema-after-data
> > switches.
>=20
> OK

I could go either way on this.

> Can we prune down to the base use case to avoid this overhead? i.e. have
> these options on pg_dump only?

Makes sense to me.

	Thanks,

		Stephen

--yRA+Bmk8aPhU85Qt
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkiLPo4ACgkQrzgMPqB3kijCPgCfYS0U47IxRZRUNUkMuoyNLBJZ
ZFAAnjA9ASB3r8/nX/ZW3gwv5oR/3JR8
=dUrL
-----END PGP SIGNATURE-----

--yRA+Bmk8aPhU85Qt--