From: Nathan Bossart <nathandbossart@gmail.com>
To: David Steele <david@pgmasters.net>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: remove more archiving overhead
Date: Thu, 7 Jul 2022 15:07:16 -0700
Message-ID: <20220707220716.GA2258527@nathanxps13> (raw)
In-Reply-To: <ba7e11b0-054d-1d26-bafd-7a2bc97f11f7@pgmasters.net>
References: <20220224.141349.1809640551882183816.horikyota.ntt@gmail.com>
<20220224175553.GA660690@nathanxps13>
<20220407222327.GA1363808@nathanxps13>
<768e6cb3-256d-9c0b-1797-62420ffca7ae@oss.nttdata.com>
<CA+TgmoaeTe4oUT39A4nt=8LD6UJ5u0vcmGc5+Aksn-4oKRb8-w@mail.gmail.com>
<17147b74-5465-9a7a-ae27-305d1200482c@pgmasters.net>
<20220707161825.GA2205004@nathanxps13>
<20220707175142.GA2254092@nathanxps13>
<20220707182251.GA2255242@nathanxps13>
<ba7e11b0-054d-1d26-bafd-7a2bc97f11f7@pgmasters.net>
On Thu, Jul 07, 2022 at 05:06:13PM -0400, David Steele wrote:
> On 7/7/22 14:22, Nathan Bossart wrote:
>> On Thu, Jul 07, 2022 at 10:51:42AM -0700, Nathan Bossart wrote:
>> > + library to ensure that it indeed does not overwrite an existing file. When
>> > + a pre-existing file is encountered, the archive library may return
>> > + <literal>true</literal> if the WAL file has identical contents to the
>> > + pre-existing archive. Alternatively, the archive library can return
>>
>> This should likely say something about ensuring the pre-existing file is
>> persisted to disk before returning true.
>
> Agreed.
>
> Also, I'd prefer to remove "indeed" from this sentence (yes, I see that was
> in the original text):
>
> + library to ensure that it indeed does not overwrite an existing
Here's an updated patch.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.comAttachments:v2-0001-add-note-about-re-archiving-in-docs.patch (text/x-diff, 2.5K) download
From 044962852ff86ffd768e124bff9883b6f7591b62 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathandbossart@gmail.com>
Date: Thu, 7 Jul 2022 10:43:30 -0700
Subject: [PATCH v2 1/1] add note about re-archiving in docs
---
doc/src/sgml/backup.sgml | 24 +++++++++++++++++++-----
1 file changed, 19 insertions(+), 5 deletions(-)
diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgmlindex 73a774d3d7..4462bfe7a9 100644--- a/doc/src/sgml/backup.sgml+++ b/doc/src/sgml/backup.sgml@@ -681,14 +681,28 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 && cp pg_wal/0
any pre-existing archive file. This is an important safety feature to
preserve the integrity of your archive in case of administrator error
(such as sending the output of two different servers to the same archive
- directory).+ directory). It is advisable to test your proposed archive library to ensure+ that it does not overwrite an existing file.
</para>
<para>
- It is advisable to test your proposed archive library to ensure that it- indeed does not overwrite an existing file, <emphasis>and that it returns- <literal>false</literal> in this case</emphasis>.- The example command above for Unix ensures this by including a separate+ In rare cases, <productname>PostgreSQL</productname> may attempt to+ re-archive a WAL file that was previously archived. For example, if the+ system crashes before the server makes a durable record of archival success,+ the server will attempt to archive the file again after restarting (provided+ archiving is still enabled). When an archive library encounters a+ pre-existing file, it may return <literal>true</literal> if the WAL file has+ identical contents to the pre-existing archive and the pre-existing archive+ is fully persisted to storage. Alternatively, the archive library can+ return <literal>false</literal> anytime a pre-existing file is encountered,+ but this will require manual action by an administrator to resolve. If a+ pre-existing file contains different contents than the WAL file being+ archived, the archive library <emphasis>must</emphasis> return false.+ </para>++ <para>+ The example command above for Unix avoids overwriting a pre-existing archive+ by including a separate
<command>test</command> step. On some Unix platforms, <command>cp</command> has
switches such as <option>-i</option> that can be used to do the same thing
less verbosely, but you should not rely on these without verifying that
--
2.25.1
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, david@pgmasters.net, robertmhaas@gmail.com, masao.fujii@oss.nttdata.com, horikyota.ntt@gmail.com
Subject: Re: remove more archiving overhead
In-Reply-To: <20220707220716.GA2258527@nathanxps13>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox