Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.6\))
Subject: Re: Optionally automatically disable logical replication
 subscriptions on error
From: Mark Dilger <mark.dilger@enterprisedb.com>
In-Reply-To: 
 <CAD21AoB+QH4or-QDiakALZp6pkXiAaGxfGcsvNYJoDHk9kknZw@mail.gmail.com>
Date: Sun, 20 Jun 2021 19:26:23 -0700
Cc: Amit Kapila <amit.kapila16@gmail.com>,
 "Smith, Peter" <peters@fast.au.fujitsu.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <F25752DE-45EC-4DDE-97D9-75F270FB67CD@enterprisedb.com>
References: <DB35438F-9356-4841-89A0-412709EBD3AB@enterprisedb.com>
 <CAA4eK1KkhsNAW3=XxOdVRD9+RcQG5PRRFBFzAMibMG0YFFmAmg@mail.gmail.com>
 <B522A0FA-0B5C-4A66-9B73-928CAA86FB88@enterprisedb.com>
 <CAA4eK1KM23D06_rUEsgnTFUpABoeCCoTun2TMrQzWgxWOYY7fA@mail.gmail.com>
 <DE7E13B7-DC76-416A-A98F-3BC3F80E6BE9@enterprisedb.com>
 <CAD21AoB+QH4or-QDiakALZp6pkXiAaGxfGcsvNYJoDHk9kknZw@mail.gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/F25752DE-45EC-4DDE-97D9-75F270FB67CD%40enterprisedb.com>
Precedence: bulk


> On Jun 20, 2021, at 7:17 PM, Masahiko Sawada <sawada.mshk@gmail.com> =
wrote:
>=20
> I will submit the patch.

Great, thanks!

> There was a discussion that the skipping transaction patch would also
> need to have a feature that tells users the details of the last
> failure transaction such as its XID, timestamp, action etc. In that
> sense, those two patches might need the common infrastructure that the
> apply workers leave the error details somewhere so that the users can
> see it.

Right.  Subscription on error triggers would need that, too, if we wrote =
them.

> Is it really useful to write only error message to the system catalog?
> Even if we see the error message like "duplicate key value violates
> unique constraint =E2=80=9Ctest_tab_pkey=E2=80=9D=E2=80=9D on the =
system catalog, we will end
> up needing to check the server log for details to properly resolve the
> conflict. If the user wants to know whether the subscription is
> disabled manually or automatically, the error message on the system
> catalog might not necessarily be necessary.

We can put more information in there.  I don't feel strongly about it.  =
I'll wait for your patch to see what infrastructure you need.

> The feature discussed in that thread is meant to be a repair tool for
> the subscription in emergency cases when something that should not
> have happened happened. I guess that resolving row (or column) level
> conflict should be done in another way, for example, by defining
> policies for each type of conflict.

I understand that is the idea, but I'm having trouble believing it will =
work that way in practice.  If somebody has a subscription that has gone =
awry, what reason do we have to believe there will only be one =
transaction that will need to be manually purged?  It seems just as =
likely that there would be a million transactions that need to be =
purged, and creating an interface for users to manually review them and =
keep or discard on a case by case basis seems unworkable.  Sure, you =
might have specific cases where the number of transactions to purge is =
small, but I don't like designing the feature around that assumption.

All the same, I'm looking forward to seeing your patch!

=E2=80=94
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company