Message-ID: <55534930.4040905@iki.fi>
Date: Wed, 13 May 2015 15:53:04 +0300
From: Heikki Linnakangas <hlinnaka@iki.fi>
Reply-To: hlinnaka@iki.fi
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:31.0) Gecko/20100101 Icedove/31.6.0
MIME-Version: 1.0
To: Robert Haas <robertmhaas@gmail.com>
CC: Michael Paquier <michael.paquier@gmail.com>,
 Venkata Balaji N <nag1010@gmail.com>,
 Andres Freund <andres@anarazel.de>, Fujii Masao <masao.fujii@gmail.com>,
 Borodin Vladimir <root@simply.name>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Streaming replication and WAL archive interactions
References: <548AF1CB.80702@vmware.com>
	<689EB259-44C2-4820-B901-4F6B1C55A1E4@simply.name>
	<549083D6.1000301@vmware.com>
	<CAHGQGwGJzp-QS7BODiv1uc291gAKtjzzCPb_nzUTxYKJhLsUCA@mail.gmail.com>
	<54949108.3030109@vmware.com>
	<CAEyp7J9Hy8Q__FbGeR5skjk7d0dvLC+KLXB3JUuWrXXdJ5O+Wg@mail.gmail.com>
	<552FA38F.9060005@iki.fi>
	<CAB7nPqQE179yogtg+nKvdwt9KROxTyt-EjumKOMbuXQtea5r3w@mail.gmail.com>
	<5535FE71.1010905@iki.fi>
	<CAB7nPqS=V=LF-JZsv9eGF84joQ3jxugt4gkOK6kD1CiB98_vWg@mail.gmail.com>
	<55362CAD.2000207@iki.fi>
	<CA+TgmoZhcbyDBr8UdGu=y5kmnVcSwcnaBqqV=UnP+oCGiNQGeQ@mail.gmail.com>
	<CAB7nPqRhsOHSc3zyL0qw4-CRphb-9Nm4sXhRCA4iKHUhkdFysA@mail.gmail.com>
	<553741FE.1080403@iki.fi>
	<CAB7nPqSak3K65B+Pzp_qGrd0rO9L=1T5FLUkPEXRqtgvV9zXEw@mail.gmail.com>
	<554CB84E.3070406@iki.fi>	<5550D20D.6090703@iki.fi>
 <CA+Tgmoa9WQjBD4Nv1Tr-eYb93_Hemz9JmJdQdrja9ocnC24X-g@mail.gmail.com>
In-Reply-To: 
 <CA+Tgmoa9WQjBD4Nv1Tr-eYb93_Hemz9JmJdQdrja9ocnC24X-g@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 05/13/2015 03:36 PM, Robert Haas wrote:
> On Mon, May 11, 2015 at 12:00 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> And here is a new version of the patch. I kept the approach of using pgstat,
>> but it now only polls pgstat every 10 seconds, and doesn't block to wait for
>> updated stats.
>
> It's not entirely a new problem, but this error message has gotten pretty crazy:
>
> +                               (errmsg("WAL archival
> (archive_mode=on/always/shared) requires wal_level \"archive\",
> \"hot_standby\", or \"logical\"")));
>
> Maybe: WAL archival cannot be enabled when wal_level is "minimal"
>
> I think the documentation should be explicit about what happens if the
> primary archives a file and dies before the standby gets notified that
> the archiving happened.

Yes, good point.

>  The standby, running in shared mode, is then
> promoted.  My first guess would be that the standby will end up with
> files that thinks it needs to archive but, being unable to do so
> because they're already there, they'll live forever in pg_xlog.  I
> hope that's not the case.

Hmm. That is exactly what happens. The standby will attempt to archive 
them, which will fail, so the archiver will get stuck retrying.

That's not actually a new problem though. Even with a single server 
doing archiving, it's possible that you crash just after archive_command 
has archived a file, but before it has created the .done file. After 
restart, the server will try to archive the file again, which will fail. 
But yeah, with this patch, that's much more likely to happen after a 
promotion.

Our manual says that archive_command should refuse to overwrite an 
existing file. But to work-around the double-archival problem, where the 
same file is archived twice, it would be even better if it would simply 
return success if the file exists, *and has identical contents*. I don't 
know how to code that logic in a simple one-liner though.

- Heikki


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers