Date: Wed, 1 Nov 2017 18:18:07 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
Cc: pgsql-admin@postgresql.org
Subject: Re: Bad recovery: no pg_xlog/RECOVERYXLOG
Message-ID: <20171101221807.GW4628@tamriel.snowman.net>
References: <d90474b1-6913-f3ff-d224-73fd931cdd98@cahoots.pl>
 <1070071508952614@web46j.yandex.ru>
 <ae7f2ee9-97ff-267e-b6dc-9399250a9314@cahoots.pl>
 <20171030154706.GE4628@tamriel.snowman.net>
 <7966ee9e-2241-eced-4639-e32c157b915a@catalyst.net.nz>
 <20171031114756.GP4628@tamriel.snowman.net>
 <2e3453d3-da6f-88af-05ec-7530e7f60547@catalyst.net.nz>
 <20171101122847.GS4628@tamriel.snowman.net>
 <4efa2769-78c4-7bf0-5c33-0cf2d942ce74@catalyst.net.nz>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="s0rzjp2JQi5zgu5F"
Content-Disposition: inline
In-Reply-To: <4efa2769-78c4-7bf0-5c33-0cf2d942ce74@catalyst.net.nz>
User-Agent: Mutt/1.5.21 (2010-09-15)
Precedence: bulk
Sender: pgsql-admin-owner@postgresql.org


--s0rzjp2JQi5zgu5F
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Mark,

* Mark Kirkwood (mark.kirkwood@catalyst.net.nz) wrote:
> However there is the use case for people that just want a minimal
> backup solution that works for their specific environment, and don't
> want to bring along a lot of extra machinery that a full coverage
> all-singing-and-dancing product includes - this *can* be
> accomplished by a few shell scripts. Yes, it does mean that you
> spend extra time testing and debugging [1]. Err - I think that is
> all the original author (who is probably scared off now), was
> wanting a bit of help with.

This is exactly the issue that concerns me.  I'm not suggesting that
these scripts are, or need to be, the end-all, be-all of PG backup
solutions.

What I'm pointing out is that shell-script based solutions are *broken*,
not that they are lacking in features.  Many, many years ago I also used
to think it was possible to perform a PG backup using just shell scripts
and have it be successful and reliable, but since then I've seen too
many cases where exactly that has lead to incomplete and invalid
backups to be able to agree that they're reasonable to use.  Not having
a way to reliably sync the WAL files copied by archive command to disk,
in particular, really is an issue, it's not some feature, it's a
requirement of a functional PG backup system.  The other requirement for
a functional PG backup system is a check to verify that all of the WAL
for a given backup has been archived safely to disk, otherwise the
backup is incomplete and can't be used.

Both of those basic requirements are, at best, extremely difficult to
do in a shell script.  Maybe it's possible to do, but I've certainly yet
to see it and I'm not going to agree that such "simple" shell scripts
should be posted to our mailing lists without someone pointing out that
they're broken because, otherwise, people will take and use them and end
up with backups that are broken (often right when they end up actually
needing it).

If you'd like to develop a shell script that addresses these basic
requirements of file-based PG backups and ask for critique on it while
making it clear that it's in development, I'd be happy to provide
comments on it.  I won't agree that any shell-based solution that
doesn't have these basic requirements met is an acceptable option.

Thanks!

Stephen

--s0rzjp2JQi5zgu5F
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJZ+kgfAAoJEO1sijiDR2RVmvcP/0kpnsPlXiH4SowYfCq3Vo4+
3F8erTh6o0yQ24MsCb1LM51DrGvuPir1EG7CLhJos5BCalag/RjSgFV8G7ArJxSx
tNFaykXpOyKkZ9cVkjYFUFo7RGtbcNxf2j8u9/eBld+vxFkO7Ko5JAO4R04dFJp3
8+Ae2Je6LPMD0RNtU0kCwsUJB6cgCiB0HKysKHOoTndVG8A0DV6qLolU3ZAJrxcU
ExgGH5qUCOSAuaLOAJ49qgGxeLghY9M2uio0GiHnH5ZZTS2UOzb5K7W6e6aEbZdS
8UEc0n+NLQ2unt79KnJeHhFA7Vtf4zlyW9hV66yTs9Gyezg9x4xD0uNj3/QqlJZC
xPxhsczZ0z5Mg9BX8iWr74NW5iRt4pxKG12hK5CV3pNDabTOXKihxQyNKFoltUmv
l4Kf3zXwb1k/4Edv9mQ0YX01VRyS0I7z5xXhmnOBRiXsBj8AThEo3U2cR5WEA7JW
kmM5kwecLHmKe95/5FXFpjgVGai9HPQCN3KXyvxTMyl0KkZRnU+4KXendaCIC6AV
yUqpOf9h1lRFGpkUfN9yAsN+5C8APAsHeWq9ryqchPk/IBamKSZPd+MEqSbx4vFk
Q50nI3KuVedXoMBvVIjY4xhspXGp3T0nhffzaiIsVyZSGVUj3WWyVtnY5+K9u7cI
Xryqxv1pex8Om9GU/lk2
=NLP9
-----END PGP SIGNATURE-----

--s0rzjp2JQi5zgu5F--