Date: Wed, 9 Mar 2022 10:42:10 -0500
From: Stephen Frost <sfrost@snowman.net>
To: Chapman Flack <chap@anastigmatix.net>
Cc: Nathan Bossart <nathandbossart@gmail.com>,
	David Steele <david@pgmasters.net>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: Postgres restart in the middle of exclusive backup and the
 presence of backup_label file
Message-ID: <20220309154209.GU10577@tamriel.snowman.net>
References: <621E6C2E.2050801@anastigmatix.net>
 <20220301191441.GW10577@tamriel.snowman.net>
 <621E7A5C.1030203@anastigmatix.net>
 <20220301200523.GX10577@tamriel.snowman.net>
 <20220302010302.GA1182956@nathanxps13>
 <621FC447.90502@anastigmatix.net>
 <20220308200107.GA85558@nathanxps13>
 <eafdd15c-6597-f205-c2df-3fb3817ccbd4@pgmasters.net>
 <20220308221253.GA169440@nathanxps13>
 <622801BD.9020605@anastigmatix.net>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="vrbhSCwwl5wYcrSs"
Content-Disposition: inline
In-Reply-To: <622801BD.9020605@anastigmatix.net>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: <https://www.postgresql.org/message-id/20220309154209.GU10577%40tamriel.snowman.net>
Precedence: bulk


--vrbhSCwwl5wYcrSs
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Chapman Flack (chap@anastigmatix.net) wrote:
> On 03/08/22 17:12, Nathan Bossart wrote:
> > I spent some time trying to come up with a workable script to replace t=
he
> > existing one.  I think the main problem is that you need to write out b=
oth
> > the backup label file and the tablespace map file, but I didn't find an
> > easy way to write the different output columns of pg_backup_stop() to
> > separate files via psql.

Let's not confuse ourselves here- the existing script *doesn't* work in
any reasonable way when we're talking about everything that needs to be
done to perform a backup.  That a lot of people are using it because
it's in the documentation is an actively bad thing.

The same goes for the archive command example.

> Something like this might work:
>=20
> SELECT * FROM pg_backup_stop(true) \gset
>=20
> \out /tmp/backup_label   \qecho :labelfile
> \out /tmp/tablespace_map \qecho :spcmapfile
> \out
> \! ... tar command adding /tmp/{backup_label,tablespace_map} to the tarba=
ll

=2E.. this doesn't do what's needed either.  We could try to write down
some minimum set of things that are needed for a backup tool to do but
it's not something that a 3 line script is going to cover.  Indeed, it's
a lot more like pg_basebackup and if we want to give folks a script to
use, it should be "run pg_basebackup".

> I notice the \qecho adds a final newline (and so if :spcmapfile is empty,
> a file containing a single newline is made). In a quick test with a bogus
> restore_command, I did not see any error messages specific to the format
> of the backup_label or tablespace_map files, so maybe the final newline
> isn't a problem.
>=20
> Assuming the newline isn't a problem, that might be simple enough to
> use in an example, and maybe it's not a bad thing that it highlights a few
> psql capabilities the reader might not have stumbled on before. Or, maybe
> it is just too confusing to bother.

It's more than just too confusing, it's actively bad because people will
actually use it and then end up with backups that don't work.

> While agreeing that pg_basebackup is the production-ready thing that
> does it all for you (with tests for likely errors and so on), I think
> there is also some value in a dead-simple example that concretely
> shows you what "it" is, what the basic steps are that happen beneath
> pg_basebackup's chrome.

Documenting everything that pg_basebackup does to make sure that the
backup is viable might be something to work on if someone is really
excited about this, but it's not 'dead-simple' and it's darn close to
the bare minimum, something that none of these simple scripts will come
anywhere close to being and instead they'll be far less than the
minimum.

> If the added newline is a problem, I haven't thought of a way to exclude
> it that doesn't take the example out of the realm of dead-simple.

I disagree that there's really a way to provide 'dead-simple' backups
with what's built into core without using pg_basebackup.  If we want a
'dead-simple' solution in core then we'd need to write an appropriate
backup tool that does all the basic things and include and maintain
that.  Writing a shell script isn't enough and we shouldn't encourage
our users to do exactly that by having it in our documentation because
then they'll think it's enough.

Thanks,

Stephen

--vrbhSCwwl5wYcrSs
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJiKMrRAAoJEO1sijiDR2RVfxcP/jEacvp8Glfgk8DoQJ0e2EKj
iDBP0DqkPEgNs9mQbvn7/Q3gMkb1Vo0P4/tSErSnDyuUT7v9I7Kf8AwkhMzLJEg1
i4VI1FHBD1glz0W0bSh14306tDTkYZ5WJ1do8Tj+CapreGIqN7OChiY1GyQIrKvD
HFUTXAqsds+DstUcubdNbHxq1epcHXo4SUkdX0mkHDlyzvKSXCbzKEQafYR27RVm
hSt4xJdkBaYvWMQCPsr+qctjjblHDtApuKDBPN8iFlnHgOQKcOJxpxr5SyMGpVg8
Qh3jjW7pFUEx+8Xragw+fP/IOew66+7SaspBTxpW8WRtuCT6YZKtY/kIxDlTOcJN
fPDEnizC87NvMLre5o1NSLdGHcsxMugF/MIYxLxscvAojDKGL3kWRS6sdqfAcIPF
NubuAztBxtNLfDWKB5iWOnDTfiX+1w7Ty92S9lJxPGboECQfOS13QTQ90UIvWfjg
4m8tJlJ4iBe1iITdczL3cy1kUIOYcC0OPb6b9wji9kzQqjvHAsjpN6DAc0LaSQUc
J71Tz3lTRT3vfHKWiWu76ZZn6oM/jGBNmVgbLyHCSoNuv6wKmoZazke7rdyzJiw9
FoZ7lBr91F4hwmQ2UGTwk7z8mE4cqFKCgwQajntsth7Rq7GQKwgjaIKbQcL0HUPS
Q6C29Ru+pRpCWpzoHk4T
=FvwP
-----END PGP SIGNATURE-----

--vrbhSCwwl5wYcrSs--