From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Avoid creating archive status ".ready" files too early
Date: Mon, 23 Aug 2021 19:54:12 +0000
Message-ID: <E1mIG1M-0001d1-Jz@gemulon.postgresql.org> (raw)
Avoid creating archive status ".ready" files too early
WAL records may span multiple segments, but XLogWrite() does not
wait for the entire record to be written out to disk before
creating archive status files. Instead, as soon as the last WAL page of
the segment is written, the archive status file is created, and the
archiver may process it. If PostgreSQL crashes before it is able to
write and flush the rest of the record (in the next WAL segment), the
wrong version of the first segment file lingers in the archive, which
causes operations such as point-in-time restores to fail.
To fix this, keep track of records that span across segments and ensure
that segments are only marked ready-for-archival once such records have
been completely written to disk.
This has always been wrong, so backpatch all the way back.
Author: Nathan Bossart <bossartn@amazon.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Ryo Matsumura <matsumura.ryo@fujitsu.com>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/CBDDFA01-6E40-46BB-9F98-9340F4379505@amazon.com
Branch
------
REL9_6_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/e85f00501e16f39f32d48c83d38634e93b69d7cd
Modified Files
--------------
src/backend/access/transam/xlog.c | 216 +++++++++++++++++++++++++++++++++++--
src/backend/postmaster/walwriter.c | 7 ++
src/include/access/xlog.h | 1 +
src/include/access/xlogdefs.h | 1 +
4 files changed, 215 insertions(+), 10 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-committers@postgresql.org
Cc: alvherre@alvh.no-ip.org
Subject: Re: pgsql: Avoid creating archive status ".ready" files too early
In-Reply-To: <E1mIG1M-0001d1-Jz@gemulon.postgresql.org>
References: <E1mIG1M-0001d1-Jz@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox