Subject: Why clear pg_wal when recovering?

From: "Mikael Sterner" <[email-masked]>
Date: Fri, 12 Nov 2021 16:55:48 +0100

Hi!

In the guide at https://www.postgresql.org/docs/current/continuous-archiving.html#BACKUP-PITR-RECOVERY
step 5 is to remove any files present in the pg_wal directory. Is this
still the recommendation if the source of the file system restore in step 4
was a base backup generated by pg_basebackup? Because in that case, wouldn't
the files in pg_wal still be needed to recover from the basebackup state?

I guess these files that were removed would normally be restored from the
WAL archive, but if you have for some reason lost your WAL archive, or it's
not complete, wouldn't it be better if you had retained them in the restored
data directory so that you could at least reach the state at the end of your
latest base backup?

Yours,
Mikael

---

From: Tom Lane <[email-masked]>
Date: Sun, 14 Nov 2021 13:13:39 -0500

[ moving to -docs ]

> the files in pg_wal still be needed to recover from the basebackup state?

I think you are right.  This text was written on the assumption that you
are restoring from a generic file system backup (made per the directions
in 26.2).  Somebody shoved it under 26.3, where it looks like it's meant
to describe recovery starting from a pg_basebackup backup, and did not
change the text.

In the generic-backup case, it definitely is a good idea to remove
whatever's in pg_wal, because some of those files are going to be future
WAL files that were just renamed into place and don't yet contain valid
data.  They'd be okay if you just want to do a PITR recovery up to the
instant when the backup was taken, but they're NOT okay for running
forward from there.  You want to flush them so the server will fetch
valid versions from the archive.  I am not totally sure, but I think
that basebackup is smarter and doesn't send any not-yet-valid WAL
files.

So I see two questions here:

1. Should we change step 5 to say "only do this when recovering from a
generic backup"?  Needs a bit of research into exactly what
pg_basebackup puts into pg_wal/.

2. Should we restructure chapter 26 to pull the how-to-recover
directions into a separate <sect1>?  If not, we probably at least
need a pointer from 26.2 to 26.3.4.

			regards, tom lane