Date: Fri, 14 Dec 2018 10:56:45 -0500
From: Stephen Frost <sfrost@snowman.net>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: =?iso-8859-1?Q?Fabr=EDzio?= de Royes Mello <fabriziomello@gmail.com>,
	petr.jelinek@2ndquadrant.com,
	Euler Taveira de Oliveira <euler@timbira.com.br>,
	Pgsql Hackers <pgsql-hackers@postgresql.org>, hironobu@interdb.jp
Subject: Re: row filtering for logical replication
Message-ID: <20181214155645.GV3415@tamriel.snowman.net>
References: 
 <CAHE3wggb715X+mK_DitLXF25B=jE6xyNCH4YOwM860JR7HarGQ@mail.gmail.com>
 <CAHE3wgiHOdFHcEpm_w5f8vAXiugDmTciTiegNGVktezMbFkkrg@mail.gmail.com>
 <7106a0fc-8017-c0fe-a407-9466c9407ff8@2ndquadrant.com>
 <CAHE3wggWAtbT+Yt_MDy0uQwZcc6x820OwHp3_TBtAqtBeW4+Ug@mail.gmail.com>
 <92e5587d-28b8-5849-2374-5ca3863256f1@2ndquadrant.com>
 <CAFcNs+q88Z--uHbcii80r-pBWByEVNbc1nPyN8iHHKhCHCtzrA@mail.gmail.com>
 <466cdec7-92a0-41c0-b74a-ec0be94d4451@2ndquadrant.com>
 <CAFcNs+pA9ywYJbf940UFXW5B5i+An3=0yzxC8_bXubu+CsOQ=A@mail.gmail.com>
 <20181123190330.GI3415@tamriel.snowman.net>
 <b2f86a5b-06fa-d1a9-1295-db673ebed550@2ndquadrant.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="ys5u3zEGNU0GrflS"
Content-Disposition: inline
In-Reply-To: <b2f86a5b-06fa-d1a9-1295-db673ebed550@2ndquadrant.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk


--ys5u3zEGNU0GrflS
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Tomas Vondra (tomas.vondra@2ndquadrant.com) wrote:
> On 11/23/18 8:03 PM, Stephen Frost wrote:
> > * Fabr=EDzio de Royes Mello (fabriziomello@gmail.com) wrote:
> >> On Fri, Nov 23, 2018 at 4:13 PM Petr Jelinek <petr.jelinek@2ndquadrant=
=2Ecom>
> >> wrote:
> >>>> If carefully documented I see no problem with it... we already have =
an
> >>>> analogous problem with functional indexes.
> >>>
> >>> The difference is that with functional indexes you can recreate the
> >>> missing object and everything is okay again. With logical replication
> >>> recreating the object will not help.
> >>>
> >>
> >> In this case with logical replication you should rsync the object. Tha=
t is
> >> the price of misunderstanding / bad use of the new feature.
> >>
> >> As usual, there are no free beer ;-)
> >=20
> > There's also certainly no shortage of other ways to break logical
> > replication, including ways that would also be hard to recover from
> > today other than doing a full resync.
>=20
> Sure, but that seems more like an argument against creating additional
> ones (and for preventing those that already exist). I'm not sure this
> particular feature is where we should draw the line, though.

I was actually going in the other direction- we should allow it because
advanced users may know what they're doing better than we do and we
shouldn't prevent things just because they might be misused or
misunderstood by a user.

> > What that seems to indicate, to me at least, is that it'd be awful
> > nice to have a way to resync the data which doesn't necessairly
> > involve transferring all of it over again.
> >=20
> > Of course, it'd be nice if we could track those dependencies too,
> > but that's yet another thing.
>=20
> Yep, that seems like a good idea in general. Both here and for
> functional indexes (although I suppose sure is a technical reason why it
> wasn't implemented right away for them).

We don't track function dependencies in general and I could certainly
see cases where you really wouldn't want to do so, at least not in the
same way that we track FKs or similar.  I do wonder if maybe we didn't
track function dependencies because we didn't (yet) have create or
replace function and that now we should.  We don't track dependencies
inside a function either though.

> > In short, I'm not sure that I agree with the idea that we shouldn't
> > allow this and instead I'd rather we realize it and put the logical
> > replication into some kind of an error state that requires a resync.
>=20
> That would still mean a need to resync the data to recover, so I'm not
> sure it's really an improvement. And I suppose it'd require tracking the
> dependencies, because how else would you mark the subscription as
> requiring a resync? At which point we could decline the DROP without a
> CASCADE, just like we do elsewhere, no?

I was actually thinking more along the lines of just simply marking the
publication/subscription as being in a 'failed' state when a failure
actually happens, and maybe even at that point basically throwing away
everything except the shell of the publication/subscription (so the user
can see that it failed and come in and properly drop it); I'm thinking
about this as perhaps similar to a transaction being aborted.

Thanks!

Stephen

--ys5u3zEGNU0GrflS
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJcE9K9AAoJEO1sijiDR2RVllIP/RkZ8v2BUel38ksiHjKKt2mQ
6cdm4QWEiguggFafIQFIYlRHrFB2wnTSTvleXw9RvVWMWXNgY4wnqkrZTPdHHkFN
r2ZblGtBUcP27Cgiay4AUiLtRiWo33bKx+d95pCGwCX7Q27lhqCW63Y8pVifyoha
cefmRW27rWasvpv90GNpkxoYF0iUqoAFQD1/7N5M/lJXKmrCWGML3gZZgRL4Sss/
Z0TJr2bGJnzafM/wWTYqwwyMUECDKB212YMXdcKMzOunlnZ5IbJPtOF2wmELNGmk
xzTtfFKv51sPYeMqOIkQHUNtfQwfFNMRhyPHBg6aNVQ0KcCsjwkxepU0xf577gdx
k2X9vbSof8Yz0drHwmfA5m4e75vzk1la+3dWZVr7yQ98tqm6Ti9QXlmbvQjtKfh4
Fe1QxAQy2bdt8noEZpm//GRXyS9uvw9pACZhHqL4MoiMrvzPsRxFlGbz8UdyYsUJ
LOGdJuQiixi2LV+X0LYh3YULu3Mx0YV8L1la1utuDRvrjQEDmroTj6OKsjatF27j
wlbtey49nwH3z74k7nvAoCAveDs16n7biqr5dKuxHqlSSIbAQpcoNfFORgN99VVx
Ph17jjcFyX8iOC4a9hXnQoRHvkWTVye/TtHN68ULmayU8TrphWMszBOKUbJhPvLS
zVRSIVFisFVwhhomSFyJ
=t2WS
-----END PGP SIGNATURE-----

--ys5u3zEGNU0GrflS--