public inbox for [email protected]  

help / color / mirror / Atom feed

From: Kevin Grittner <[email protected]>
To: Josh Berkus <[email protected]>
To: [email protected] <[email protected]>
Subject: Re: Sample archive_command is still problematic
Date: Mon, 11 Aug 2014 10:21:26 -0700
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
List-Unsubscribe: <mailto:[email protected]?body=unsub%20pgsql-docs>

Josh Berkus <[email protected]> wrote:

> The example archive_command we give in the docs is this one:
>
> archive_command = 'test ! -f /mnt/server/archivedir/%f && cp %p /mnt/server/archivedir/%f'
>
> This is a problematic recommendation.

I agree with that statement, ...

> If there's any reason why copying the archive file gets
> interrupted (storage blip, for example), then the command will
> fail and will continue to fail forever, ending archiving.

... but not for that reason.

> Is there some good reason why "test ! -f" was added to the
> sample?

In an environment with more than one cluster archiving, it is
otherwise way too easy to copy a config file and have the WAL files
of the two systems overwriting one another.  I consider a check for
an already existing file on the target to be very good practice.
The errors in the log are a clue that something went wrong, and
gives you a chance to fix things without data loss.

The problem with the recommended command is that cp is not atomic.
The file can be read before the contents are materialized, causing
early end to recovery.  I have seen it happen.  The right way to do
this is to copy to a different name or directory and mv the file
into place once it is complete -- or use software which does that
automatically, like rsync does.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-docs mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs

view thread (25+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: Sample archive_command is still problematic
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox