Date: Tue, 27 Nov 2018 10:34:05 -0500
From: Stephen Frost <sfrost@snowman.net>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: David Steele <david@pgmasters.net>, Andres Freund <andres@anarazel.de>,
	Michael Paquier <michael@paquier.xyz>,
	Sergei Kornilov <sk@zsrv.org>,
	PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Subject: Re: Continue work on changes to recovery.conf API
Message-ID: <20181127153405.GX3415@tamriel.snowman.net>
References: <51a6446c-c2b6-b653-c9f5-da38e23ddd9d@2ndquadrant.com>
 <22367601542121030@iva5-d3020dc3459d.qloud-c.yandex.net>
 <2d1877b6-e8cf-a27c-4608-c40758dd0f6c@2ndquadrant.com>
 <20181121060012.GE1951@paquier.xyz>
 <d7f0860c-65d6-5346-c672-3b1116b20ac4@2ndquadrant.com>
 <20181125182415.GP3415@tamriel.snowman.net>
 <20181125203927.f66lvh55ef5aw65g@alap3.anarazel.de>
 <ec74a8ae-8d25-6d60-1db4-907c7999acbb@2ndquadrant.com>
 <0ea1b300-38b5-bd96-1fa0-784729160efa@pgmasters.net>
 <26178006-74c0-4599-0125-6bf9eb489e96@2ndquadrant.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="m65Gnd0q9NXNFThY"
Content-Disposition: inline
In-Reply-To: <26178006-74c0-4599-0125-6bf9eb489e96@2ndquadrant.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk


--m65Gnd0q9NXNFThY
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Peter Eisentraut (peter.eisentraut@2ndquadrant.com) wrote:
> On 27/11/2018 13:21, David Steele wrote:
> > I would prefer a specific file that will be auto-included into
> > postgresql.conf when present but be ignored when not present.  Some
> > settings are generally ephemeral (recovery_target_time) and it would be
> > nice for them to go away.  When recovery is complete the file would be
> > renamed to .done just as recovery.conf is now.
>=20
> That might be a useful facility, but it wouldn't really address the
> pg_basebackup -R issue, because that creates settings that you don't
> want going away in this manner.  You'd then need two separate such
> files, one for targeted recovery that goes away when recovery ends, and
> one that is automatically included that pg_basebackup can overwrite at wi=
ll.

I've been thinking about this also and I agree that there's some
challenges when it comes to having another file- what happens if someone
does an ALTER SYSTEM on primary_conninfo while it's in the
'recovery.auto.conf' (or whatever) file?  Does that go into
postgresql.auto.conf, or somewhere else?  What if they do a RESET?

Then there's the other side of things- do we really want things like
recovery_target_time being kept around in postgresql.auto.conf after a
promotoion?  Do we want to keep appending primary_conninfo into
postgresql.auto.conf?  I haven't looked but I'm also concerned that
something like ALTER SYSTEM RESET isn't really prepared to find
duplicates in postgresql.auto.conf...

Maybe thinking through what we want to have happen in each scenario
would be good.  If you perform a pg_basebackup -R and there's already
something set in postgresql.auto.conf for primary conninfo- what should
happen?  If we reach the end of recovery and promote while
postgresql.auto.conf has recovery_target_time set, what should happen?
If third-party tools want to do what pg_basebackup -R does and modify
things in postgresql.auto.conf, how should they do that?

Here's my thoughts on answers to the above:

- pg_basebackup -R should INSERT .. ON CONFLICT UPDATE the settings that
  it wants to set in postgresql.auto.conf

- When we reach the end of recovery and promote, we should DELETE from
  the postgresql.auto.conf the recovery target settings.

- External tools should either be told that they can modify
  postgresql.auto.conf and given guideance on how to do so, or we should
  provide a tool which allows them to do so (or maybe both).

As we already have a section for recovery target settings that clearly
has them as independent, hopefully this will make sense to users.  Open
to other ideas too, of course, but I don't think we can continue to just
append things to the end of postgresql.auto.conf when pg_basebackup is
run with -R.

Thanks!

Stephen

--m65Gnd0q9NXNFThY
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJb/WPtAAoJEO1sijiDR2RVqkIP/i974KB93CJqYATo2p65K2ZW
Bj1wXgW2PbWdTVaKiRGodzdxWYwbVSLfPGtGMiX1xn2/GeUUwHqoLb3sHybE0hCX
8YYbv1KlBHz9tw445OFFtIsSZ1OMW3q02/grs9B7kshI/+IhAk8H9F7VRlPnhTbo
kD/1RVbXNCzWDqUdvHmfxOen7po3FCXcLp3zSs4/lQ/l/fLFzl53AQokf25379So
2pazTtq/v9aUaph2kNcLqef83TCIcnw2DlvDDA+feuZjVIfoRE/KEhrOBsflhoII
1Op2lLONUqmr5c330UZA0c683yAqWWkzfivPgRoW/sbtVN2O+IIVsw0fVZNzFfZg
JidTHjJaCB5cta4Xv8R4IGgSYPLiFeIzJoxfbR5QbD7i/Dab4sZBrVwqhNTwZv74
o4wpo6D66wZ9vXV+lR4cfqPjGNjH9mQ77jmesNJShiu9MX9sZcnCO1HAG6X4v2+6
JrUJHBzwR344uXCM25im6+ESh2FADYohkgbrQ4M83X5scopcRf79cT4TVlSq/pgD
/qyaaS2pda/E4aYXW2/a6R95jY037OuzvxONlPnElB5psaYuZXTJBH2n+Jf6p+sG
x3pD7z6Ngy38z2p2S/T8dSlSqV93GuGT8+Whuuu6OA6Gbl3OrqmnEPs6oX2ReG5B
xeTJKLQC7GBQtYFgZHVj
=9XAB
-----END PGP SIGNATURE-----

--m65Gnd0q9NXNFThY--