Message-ID: <553741FE.1080403@iki.fi>
Date: Wed, 22 Apr 2015 09:38:54 +0300
From: Heikki Linnakangas <hlinnaka@iki.fi>
Reply-To: hlinnaka@iki.fi
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:31.0) Gecko/20100101 Icedove/31.6.0
MIME-Version: 1.0
To: Michael Paquier <michael.paquier@gmail.com>,
 Robert Haas <robertmhaas@gmail.com>
CC: Venkata Balaji N <nag1010@gmail.com>, Andres Freund <andres@anarazel.de>,
 Fujii Masao <masao.fujii@gmail.com>, Borodin Vladimir <root@simply.name>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Streaming replication and WAL archive interactions
References: <548AF1CB.80702@vmware.com>
	<689EB259-44C2-4820-B901-4F6B1C55A1E4@simply.name>
	<549083D6.1000301@vmware.com>
	<CAHGQGwGJzp-QS7BODiv1uc291gAKtjzzCPb_nzUTxYKJhLsUCA@mail.gmail.com>
	<54949108.3030109@vmware.com>
	<CAEyp7J9Hy8Q__FbGeR5skjk7d0dvLC+KLXB3JUuWrXXdJ5O+Wg@mail.gmail.com>
	<552FA38F.9060005@iki.fi>
	<CAB7nPqQE179yogtg+nKvdwt9KROxTyt-EjumKOMbuXQtea5r3w@mail.gmail.com>
	<5535FE71.1010905@iki.fi>
	<CAB7nPqS=V=LF-JZsv9eGF84joQ3jxugt4gkOK6kD1CiB98_vWg@mail.gmail.com>
	<55362CAD.2000207@iki.fi>
	<CA+TgmoZhcbyDBr8UdGu=y5kmnVcSwcnaBqqV=UnP+oCGiNQGeQ@mail.gmail.com>
 <CAB7nPqRhsOHSc3zyL0qw4-CRphb-9Nm4sXhRCA4iKHUhkdFysA@mail.gmail.com>
In-Reply-To: 
 <CAB7nPqRhsOHSc3zyL0qw4-CRphb-9Nm4sXhRCA4iKHUhkdFysA@mail.gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 04/22/2015 03:30 AM, Michael Paquier wrote:
> This is going to change a behavior that people are used to for a
> couple of releases. I would not mind having this patch do
> "archive_mode = on during recovery" => archive only segments generated
> by this node + the last partial segment on the old timeline at
> promotion, without renaming to preserve backward compatible behavior.
> If master and standby point to separate archive locations, this way
> the operator can sort out later the one he would want to use. If they
> point to the same location, archive_command scripts already do
> internally such renaming, at least that's what I suspect an
> experienced user would do, now it is true that not many people are
> experienced in this area I see mistakes regarding such things on a
> weekly basis... This .partial segment renaming is something that we
> should let the archive_command manage with its internal logic.

Currently, the archive command doesn't know if the segment it's 
archiving is partial or not, so you can't put any logic there to manage 
it. But if we archive it with the .partial suffix, then you can put 
logic in the restore_command to check for .partial files, if you really 
want to.

I feel that the best approach is to archive the last, partial segment, 
but with the .partial suffix. I don't see any plausible real-world setup 
where the current behaviour would be better. I don't really see much 
need to archive the partial segment at all, but there's also no harm in 
doing it, as long as it's clearly marked with the .partial suffix.

BTW, pg_receivexlog also uses a ".partial" file, while it's streaming 
WAL from the server. The .partial suffix is removed when the segment is 
complete. So there's some precedence to this. pg_receivexlog adds just 
".partial" to the filename, it doesn't add any information of what 
portion of the file is valid like I suggested here, though. Perhaps we 
should follow pg_receivexlog's example at promotion too, for consistency.

- Heikki


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers