Date: Thu, 7 Jul 2022 15:07:16 -0700
From: Nathan Bossart <nathandbossart@gmail.com>
To: David Steele <david@pgmasters.net>
Cc: Robert Haas <robertmhaas@gmail.com>,
	Fujii Masao <masao.fujii@oss.nttdata.com>,
	Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: remove more archiving overhead
Message-ID: <20220707220716.GA2258527@nathanxps13>
References: <20220224.141349.1809640551882183816.horikyota.ntt@gmail.com>
 <20220224175553.GA660690@nathanxps13>
 <20220407222327.GA1363808@nathanxps13>
 <768e6cb3-256d-9c0b-1797-62420ffca7ae@oss.nttdata.com>
 <CA+TgmoaeTe4oUT39A4nt=8LD6UJ5u0vcmGc5+Aksn-4oKRb8-w@mail.gmail.com>
 <17147b74-5465-9a7a-ae27-305d1200482c@pgmasters.net>
 <20220707161825.GA2205004@nathanxps13>
 <20220707175142.GA2254092@nathanxps13>
 <20220707182251.GA2255242@nathanxps13>
 <ba7e11b0-054d-1d26-bafd-7a2bc97f11f7@pgmasters.net>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="3V7upXqbjpZ4EhLz"
Content-Disposition: inline
In-Reply-To: <ba7e11b0-054d-1d26-bafd-7a2bc97f11f7@pgmasters.net>
Archived-At: 
 <https://www.postgresql.org/message-id/20220707220716.GA2258527%40nathanxps13>
Precedence: bulk


--3V7upXqbjpZ4EhLz
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Thu, Jul 07, 2022 at 05:06:13PM -0400, David Steele wrote:
> On 7/7/22 14:22, Nathan Bossart wrote:
>> On Thu, Jul 07, 2022 at 10:51:42AM -0700, Nathan Bossart wrote:
>> > +    library to ensure that it indeed does not overwrite an existing file.  When
>> > +    a pre-existing file is encountered, the archive library may return
>> > +    <literal>true</literal> if the WAL file has identical contents to the
>> > +    pre-existing archive.  Alternatively, the archive library can return
>> 
>> This should likely say something about ensuring the pre-existing file is
>> persisted to disk before returning true.
> 
> Agreed.
> 
> Also, I'd prefer to remove "indeed" from this sentence (yes, I see that was
> in the original text):
> 
> +    library to ensure that it indeed does not overwrite an existing

Here's an updated patch.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

--3V7upXqbjpZ4EhLz
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
	filename="v2-0001-add-note-about-re-archiving-in-docs.patch"

From 044962852ff86ffd768e124bff9883b6f7591b62 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathandbossart@gmail.com>
Date: Thu, 7 Jul 2022 10:43:30 -0700
Subject: [PATCH v2 1/1] add note about re-archiving in docs

---
 doc/src/sgml/backup.sgml | 24 +++++++++++++++++++-----
 1 file changed, 19 insertions(+), 5 deletions(-)

diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml
index 73a774d3d7..4462bfe7a9 100644
--- a/doc/src/sgml/backup.sgml
+++ b/doc/src/sgml/backup.sgml
@@ -681,14 +681,28 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
     any pre-existing archive file.  This is an important safety feature to
     preserve the integrity of your archive in case of administrator error
     (such as sending the output of two different servers to the same archive
-    directory).
+    directory).  It is advisable to test your proposed archive library to ensure
+    that it does not overwrite an existing file.
    </para>
 
    <para>
-    It is advisable to test your proposed archive library to ensure that it
-    indeed does not overwrite an existing file, <emphasis>and that it returns
-    <literal>false</literal> in this case</emphasis>.
-    The example command above for Unix ensures this by including a separate
+    In rare cases, <productname>PostgreSQL</productname> may attempt to
+    re-archive a WAL file that was previously archived.  For example, if the
+    system crashes before the server makes a durable record of archival success,
+    the server will attempt to archive the file again after restarting (provided
+    archiving is still enabled).  When an archive library encounters a
+    pre-existing file, it may return <literal>true</literal> if the WAL file has
+    identical contents to the pre-existing archive and the pre-existing archive
+    is fully persisted to storage.  Alternatively, the archive library can
+    return <literal>false</literal> anytime a pre-existing file is encountered,
+    but this will require manual action by an administrator to resolve.  If a
+    pre-existing file contains different contents than the WAL file being
+    archived, the archive library <emphasis>must</emphasis> return false.
+   </para>
+
+   <para>
+    The example command above for Unix avoids overwriting a pre-existing archive
+    by including a separate
     <command>test</command> step.  On some Unix platforms, <command>cp</command> has
     switches such as <option>-i</option> that can be used to do the same thing
     less verbosely, but you should not rely on these without verifying that
-- 
2.25.1


--3V7upXqbjpZ4EhLz--