MIME-Version: 1.0
In-Reply-To: <553741FE.1080403@iki.fi>
References: <548AF1CB.80702@vmware.com>
 <689EB259-44C2-4820-B901-4F6B1C55A1E4@simply.name>
 <549083D6.1000301@vmware.com>
 <CAHGQGwGJzp-QS7BODiv1uc291gAKtjzzCPb_nzUTxYKJhLsUCA@mail.gmail.com>
 <54949108.3030109@vmware.com>
 <CAEyp7J9Hy8Q__FbGeR5skjk7d0dvLC+KLXB3JUuWrXXdJ5O+Wg@mail.gmail.com>
 <552FA38F.9060005@iki.fi>
 <CAB7nPqQE179yogtg+nKvdwt9KROxTyt-EjumKOMbuXQtea5r3w@mail.gmail.com>
 <5535FE71.1010905@iki.fi>
 <CAB7nPqS=V=LF-JZsv9eGF84joQ3jxugt4gkOK6kD1CiB98_vWg@mail.gmail.com>
 <55362CAD.2000207@iki.fi>
 <CA+TgmoZhcbyDBr8UdGu=y5kmnVcSwcnaBqqV=UnP+oCGiNQGeQ@mail.gmail.com>
 <CAB7nPqRhsOHSc3zyL0qw4-CRphb-9Nm4sXhRCA4iKHUhkdFysA@mail.gmail.com>
 <553741FE.1080403@iki.fi>
Date: Wed, 22 Apr 2015 16:07:17 +0900
Message-ID: 
 <CAB7nPqSak3K65B+Pzp_qGrd0rO9L=1T5FLUkPEXRqtgvV9zXEw@mail.gmail.com>
Subject: Re: Streaming replication and WAL archive interactions
From: Michael Paquier <michael.paquier@gmail.com>
To: hlinnaka@iki.fi
Cc: Robert Haas <robertmhaas@gmail.com>, Venkata Balaji N <nag1010@gmail.com>,
 Andres Freund <andres@anarazel.de>, Fujii Masao <masao.fujii@gmail.com>,
 Borodin Vladimir <root@simply.name>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
Content-Type: text/plain; charset=ISO-8859-1
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On Wed, Apr 22, 2015 at 3:38 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> On 04/22/2015 03:30 AM, Michael Paquier wrote:
>>
>> This is going to change a behavior that people are used to for a
>> couple of releases. I would not mind having this patch do
>> "archive_mode = on during recovery" => archive only segments generated
>> by this node + the last partial segment on the old timeline at
>> promotion, without renaming to preserve backward compatible behavior.
>> If master and standby point to separate archive locations, this way
>> the operator can sort out later the one he would want to use. If they
>> point to the same location, archive_command scripts already do
>> internally such renaming, at least that's what I suspect an
>> experienced user would do, now it is true that not many people are
>> experienced in this area I see mistakes regarding such things on a
>> weekly basis... This .partial segment renaming is something that we
>> should let the archive_command manage with its internal logic.
>
>
> Currently, the archive command doesn't know if the segment it's archiving is
> partial or not, so you can't put any logic there to manage it. But if we
> archive it with the .partial suffix, then you can put logic in the
> restore_command to check for .partial files, if you really want to.

Well, now you can check as well if there is a file with the same name
already archived and append a suffix to the new file copied, keep the
two files, and then let restore_command sort things up as it wants
with the two segment files it finds.

> I feel that the best approach is to archive the last, partial segment, but
> with the .partial suffix. I don't see any plausible real-world setup where
> the current behavior would be better. I don't really see much need to
> archive the partial segment at all, but there's also no harm in doing it, as
> long as it's clearly marked with the .partial suffix.

Well, as long as it is clearly archived at promotion, even with a
suffix, I guess that I am fine... This will need some tweaking on
restore_command for existing applications, but as long as it is
clearly documented I am fine. Shouldn't this be a different patch
though?

> BTW, pg_receivexlog also uses a ".partial" file, while it's streaming WAL
> from the server. The .partial suffix is removed when the segment is
> complete. So there's some precedence to this. pg_receivexlog adds just
> ".partial" to the filename, it doesn't add any information of what portion
> of the file is valid like I suggested here, though. Perhaps we should follow
> pg_receivexlog's example at promotion too, for consistency.

Consistency here sounds good to me.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers