From: Nagy László Zsolt <gandalf@shopzeus.com>
To: pgsql-admin@lists.postgresql.org
Subject: Re: Reliable WAL file shipping over unreliable network
Date: Wed, 28 Feb 2018 21:52:11 +0100
Message-ID: <173b7d0a-d3e4-5f1a-72ef-4b4cd6429346@shopzeus.com> (raw)
In-Reply-To: <1519848731.2350.5.camel@cybertec.at>
References: <dd116172-3ddf-ee56-57f3-6e18d49abf8e@shopzeus.com>
<1519848731.2350.5.camel@cybertec.at>
>> Do I have to copy
>> segments to temp files, and rename them when they are fully flushed to
>> disk? Or is it okay to have half complete files in the archive dir for a
>> while?
> I suppose you are talking about "archive_command" here.
>
> If the file restored with "restore_command" is too small,
> the operation fails, and you get a DEBUG1 message:
>
> archive file "..." has wrong size: ... instead of ...
>
> So nothing can go wrong there.
Nothing can go wrong? Does it mean that PostgreSQL will re-execute the
restore_command if the file was too small? If it won't retry the
restore_command, then everything goes wrong. It might be documented
somewhere, but apparently I'm out of luck with documentations.
>
>> And finally: if I also enable streaming replication, then it seems that
>> log file shipping is not needed at all. If I omit archive_command and
>> restore_command from the configs, and setup the replication slots and
>> primary_conninfo only, then it seems to be working just fine. But when
>> the network goes down for a while, then the slave goes out of sync and
>> it cannot recover. It was not clear for me from the documentation, but
>> am I right in that I can combine log file shipping with streaming
>> replication, and achieve small replication delays plus the ability to
>> recover after a longer period if network outage?
> If you use a replication slot, the standby will never get out of sync
> because the primary will retain all WAL that the standby has not
> received yet.
>
> Streaming replication together with archive recovery is only useful
> if you are *not* using replication slots.
So you are saying that if I use replication slots, then I can completely
forget about manual WAL file shipping. There is one thing in the docs
that contradicts the above statement.
This is from
https://www.postgresql.org/docs/10/static/warm-standby.html#STREAMING-REPLICATION> If you use streaming replication without file-based continuous
> archiving, the server might recycle old WAL segments before the
> standby has received them. If this occurs, the standby will need to be
> reinitialized from a new base backup.
In other words, using file based archive recovery plus streaming
replication at the same time can be useful, because it allows you you to
store WAL segments on a disk that is independent of the master, and it
makes recovery possible after a longer period of time. Increasing
wal_keep_segments may help the replication slave to recover after a
longer time, but it is more risky. The segments are stored on the
master, and they don't provide protection against disk failures (on the
master).
Cheers,
Laszlo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: gandalf@shopzeus.com, pgsql-admin@lists.postgresql.org
Subject: Re: Reliable WAL file shipping over unreliable network
In-Reply-To: <173b7d0a-d3e4-5f1a-72ef-4b4cd6429346@shopzeus.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox