Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.6\))
Subject: Re: Optionally automatically disable logical replication
 subscriptions on error
From: Mark Dilger <mark.dilger@enterprisedb.com>
In-Reply-To: 
 <CAHut+Pv1Y0LtffP=3AnZFASwJJBGeZM7bi_vYON_hfyX=VdBJg@mail.gmail.com>
Date: Mon, 21 Jun 2021 19:29:38 -0700
Cc: Amit Kapila <amit.kapila16@gmail.com>,
 Masahiko Sawada <sawada.mshk@gmail.com>,
 "Smith, Peter" <peters@fast.au.fujitsu.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <ACC5CDA8-7318-4752-B65E-479EFFA72667@enterprisedb.com>
References: <DB35438F-9356-4841-89A0-412709EBD3AB@enterprisedb.com>
 <CAA4eK1KkhsNAW3=XxOdVRD9+RcQG5PRRFBFzAMibMG0YFFmAmg@mail.gmail.com>
 <B522A0FA-0B5C-4A66-9B73-928CAA86FB88@enterprisedb.com>
 <CAA4eK1KM23D06_rUEsgnTFUpABoeCCoTun2TMrQzWgxWOYY7fA@mail.gmail.com>
 <DE7E13B7-DC76-416A-A98F-3BC3F80E6BE9@enterprisedb.com>
 <CAD21AoB+QH4or-QDiakALZp6pkXiAaGxfGcsvNYJoDHk9kknZw@mail.gmail.com>
 <F25752DE-45EC-4DDE-97D9-75F270FB67CD@enterprisedb.com>
 <CAA4eK1+NoRbYSH1J08zi4OJ_EUMcjmxTwnmwVqZ6e_xzS0D6VA@mail.gmail.com>
 <14738A39-8000-463E-A0BA-6729930A32DA@enterprisedb.com>
 <CAA4eK1+TT7Sp2VTFAcFR+wrCWiZCi1rSC5HnffMgYAg4pdxEsw@mail.gmail.com>
 <A539C848-670E-454F-B31C-82D3CBE9F5AC@enterprisedb.com>
 <CAA4eK1K_JFSFrAkr_fgp3VX6hTSmjK=wNs4Tw8rUWHGp0+Bsaw@mail.gmail.com>
 <CAA4eK1+2hsBPVn_WD6hNMZe4hWPjxshP_OegF_Py5aEmJWrWEg@mail.gmail.com>
 <CAHut+Pv1Y0LtffP=3AnZFASwJJBGeZM7bi_vYON_hfyX=VdBJg@mail.gmail.com>
To: Peter Smith <smithpb2250@gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/ACC5CDA8-7318-4752-B65E-479EFFA72667%40enterprisedb.com>
Precedence: bulk


> On Jun 21, 2021, at 5:57 PM, Peter Smith <smithpb2250@gmail.com> =
wrote:
>=20
> #5. Document to refer to the logs. All ERROR details are already in
> the logs, and this seems to me the intuitive place to look for them.

My original motivation came from writing TAP tests to check that the =
permissions systems would properly deny the apply worker when running =
under a non-superuser role.  The idea is that the user with the =
responsibility for managing subscriptions won't have enough privilege to =
read the logs.  Whatever information that user needs (if any) must be =
someplace else.

> Searching for specific errors becomes difficult programmatically (is
> this really a problem other than complex TAP tests?).

I believe there is a problem, because I remain skeptical that these =
errors will be both existent and rare.  Either you've configured your =
system correctly and you get zero of these, or you've misconfigured it =
and you get some non-zero number of them.  I don't see any reason to =
assume that number will be small.

The best way to deal with that is to be able to tell the system what to =
do with them, like "if the error has this error code and the error =
message matches this regular expression, then do this, else do that."  =
That's why I think allowing triggers to be created on subscriptions =
makes the most sense (though is probably the hardest system being =
proposed so far.)

> But here there
> is no risk of missing or insufficient information captured in the log
> files ("but still there will be some information related to ERROR
> which we wanted the user to see unless we ask them to refer to logs
> for that." [Amit-4}).

Not only is there a problem if the user doesn't have permission to view =
the logs, but also, if we automatically disable the subscription until =
the error is manually cleared, the logs might be rotated out of =
existence before the user takes any action.  In that case, the logs will =
be entirely missing, and not even the error message will remain.  At =
least with the patch I submitted, the error message will remain, though =
I take Amit's point that there are deficiencies in handling parallel =
tablesync workers, etc.

=E2=80=94
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company