Date: Sat, 30 Jul 2022 23:51:56 -0700
From: Noah Misch <noah@leadboat.com>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: David Steele <david@pgmasters.net>,
	Kyotaro Horiguchi <horikyota.ntt@gmail.com>, robertmhaas@gmail.com,
	masao.fujii@oss.nttdata.com, pgsql-hackers@postgresql.org
Subject: Re: remove more archiving overhead
Message-ID: <20220731065156.GA3694276@rfd.leadboat.com>
References: <20220707182251.GA2255242@nathanxps13>
 <ba7e11b0-054d-1d26-bafd-7a2bc97f11f7@pgmasters.net>
 <20220707220716.GA2258527@nathanxps13>
 <20220708.105652.1551158050191356887.horikyota.ntt@gmail.com>
 <44dbf5e4-8eec-da56-daba-755128b3723e@pgmasters.net>
 <20220708165450.GA2356733@nathanxps13>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220708165450.GA2356733@nathanxps13>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: 
 <https://www.postgresql.org/message-id/20220731065156.GA3694276%40rfd.leadboat.com>
Precedence: bulk

On Fri, Jul 08, 2022 at 09:54:50AM -0700, Nathan Bossart wrote:
> Since it's okay to return true or false in the identical/persisted file
> case, I didn't think it deserved emphasis.

I think returning false is not-okay:

> --- a/doc/src/sgml/backup.sgml
> +++ b/doc/src/sgml/backup.sgml
> @@ -681,14 +681,28 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
>      any pre-existing archive file.  This is an important safety feature to
>      preserve the integrity of your archive in case of administrator error
>      (such as sending the output of two different servers to the same archive
> -    directory).
> +    directory).  It is advisable to test your proposed archive library to ensure
> +    that it does not overwrite an existing file.
>     </para>
>  
>     <para>
> -    It is advisable to test your proposed archive library to ensure that it
> -    indeed does not overwrite an existing file, <emphasis>and that it returns
> -    <literal>false</literal> in this case</emphasis>.
> -    The example command above for Unix ensures this by including a separate
> +    In rare cases, <productname>PostgreSQL</productname> may attempt to
> +    re-archive a WAL file that was previously archived.  For example, if the
> +    system crashes before the server makes a durable record of archival success,
> +    the server will attempt to archive the file again after restarting (provided
> +    archiving is still enabled).  When an archive library encounters a
> +    pre-existing file, it may return <literal>true</literal> if the WAL file has
> +    identical contents to the pre-existing archive and the pre-existing archive
> +    is fully persisted to storage.  Alternatively, the archive library may
> +    return <literal>false</literal> anytime a pre-existing file is encountered,
> +    but this will require manual action by an administrator to resolve.  If a

Inviting the administrator to resolve things is more dangerous than just
returning true.  I recommend making this text more opinionated and simpler:
libraries must return true.  Alternately, if some library has found a good
reason to return false, this paragraph could give the reason.  I don't know of
such a reason, though.

> +    pre-existing file contains different contents than the WAL file being
> +    archived, the archive library <emphasis>must</emphasis> return false.
> +   </para>