Re: Streaming replication and WAL archive interactions

public inbox for [email protected]  
help / color / mirror / Atom feed

From: Michael Paquier <[email protected]>
To: [email protected]
Cc: Venkata Balaji N <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: Fujii Masao <[email protected]>
Cc: Borodin Vladimir <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: Streaming replication and WAL archive interactions
Date: Tue, 21 Apr 2015 18:04:59 +0900
Message-ID: <CAB7nPqS=V=LF-JZsv9eGF84joQ3jxugt4gkOK6kD1CiB98_vWg@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>
	<[email protected]>
	<CAHGQGwGJzp-QS7BODiv1uc291gAKtjzzCPb_nzUTxYKJhLsUCA@mail.gmail.com>
	<[email protected]>
	<CAEyp7J9Hy8Q__FbGeR5skjk7d0dvLC+KLXB3JUuWrXXdJ5O+Wg@mail.gmail.com>
	<[email protected]>
	<CAB7nPqQE179yogtg+nKvdwt9KROxTyt-EjumKOMbuXQtea5r3w@mail.gmail.com>
	<[email protected]>
List-Unsubscribe: <mailto:[email protected]?body=unsub%20pgsql-hackers>

On Tue, Apr 21, 2015 at 4:38 PM, Heikki Linnakangas <[email protected]> wrote:

> On 04/21/2015 09:53 AM, Michael Paquier wrote:
>
>> On Thu, Apr 16, 2015 at 8:57 PM, Heikki Linnakangas wrote:
>>
>>> Oh, hang on, that's not necessarily true. On promotion, the standby
>>>
>> archives
>>
>>> the last, partial WAL segment from the old timeline. That's just wrong
>>> (http://www.postgresql.org/message-id/[email protected]), and
>>> in
>>> fact I somehow thought I changed that already, but apparently not. So
>>>
>> let's
>>
>>> stop doing that.
>>>
>>
>> Er. Are you planning to prevent the standby from archiving the last
>> partial
>> segment from the old timeline at promotion?
>>
>
> Yes.
>
>  I thought from previous discussions that we should do it as master
>> (be it crashed, burned, burried or dead) may not have the occasion to
>> do it. By preventing its archiving you close the door to the case
>> where master did not have the occasion to archive it.
>>
>
> The current situation is a mess:
>
> 1. Even though we archive the last segment in the standby, there is no
> guarantee that the master had archived all the previous segments already.
>
2. If the master is not totally dead, it might try to archive the same file
> with more WAL in it, at the same time or just afterwards, or even just
> before the standby has completed promotion. Which copy do you keep in the
> archive? Having to deal with that makes the archive_command more
> complicated.
>
> Note that even though we don't archive the partial last segment on the
> previous timeline, the same WAL is copied to the first segment on the new
> timeline. So the WAL isn't lost.
>

But if the failed master has archived those segments safely, we may need
them, no? I am not sure we can ignore a user who would want to do a PITR
with recovery_target_timeline pointing to the one of the failed master.


>
>  People may be surprised that a base backup taken from a node that has
>> archive_mode = on set (that's the case in a very large number of cases)
>> will not be able to work as-is as node startup will fail as follows:
>> FATAL:  archive_mode='on' cannot be used in archive recovery
>> HINT:  Use 'shared' or 'always' mode instead.
>>
>
> Hmm, good point.
>
>  One idea would be to simply ignore the fact that archive_mode = on on
>> nodes
>> in recovery instead of dropping an error. Note that I like the fact that
>> it
>> drops an error as that's clear, I just point the fact that people may be
>> surprised that base backups are not working anymore now in this case.
>>
>
> By "ignore", what behaviour do you mean? Would "on" be equivalent to
> "shared", "always", or something else?
>

I meant something backward-compatible, with files marked as .done when they
are finished replaying... But now my words *are* weird as on != off ;)

Or we could keep the current behaviour with archive_mode=on (except for the
> last segment thing, which is just wrong), where the standby only archives
> the new timeline, and nothing from the previous timelines.
>

I guess this would solve the issue here then, which is not a bad thing in
itself:
http://www.postgresql.org/message-id/20140918180734.361021e1@erg
We would need to check if the situation improves with the 'always' mode btw.


> Are the use cases where you'd want that, rather than the new "shared"
> mode? I wanted to keep the 'on' mode for backwards-compatibility, but if
> that causes more problems, it might be better to just remove it and force
> the admin to choose what kind of a setup he has, with "shared" or "always".
>

The 'on' mode is still useful IMO to get a behavior a maximum close to what
previous releases did.
Regards,
-- 
Michael

view thread (31+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Streaming replication and WAL archive interactions
  In-Reply-To: <CAB7nPqS=V=LF-JZsv9eGF84joQ3jxugt4gkOK6kD1CiB98_vWg@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox