Received: from localhost (unknown [200.46.204.187]) by postgresql.org (Postfix) with ESMTP id 9C0F69F9684 for ; Sun, 2 Sep 2007 18:05:06 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.187]) (amavisd-maia, port 10024) with ESMTP id 12694-01 for ; Sun, 2 Sep 2007 18:04:46 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from smtp.nildram.co.uk (smtp.nildram.co.uk [195.112.4.54]) by postgresql.org (Postfix) with ESMTP id A2F599F9605 for ; Sun, 2 Sep 2007 18:05:01 -0300 (ADT) Received: from [192.168.0.3] (84-12-26-189.dyn.gotadsl.co.uk [84.12.26.189]) by smtp.nildram.co.uk (Postfix) with ESMTP id 15C0E2BBEB7; Sun, 2 Sep 2007 22:04:56 +0100 (BST) Subject: Re: pitr replica dies on startup From: Simon Riggs To: Tom Lane Cc: Jeff Frost , pgsql-admin@postgresql.org In-Reply-To: <5916.1188611810@sss.pgh.pa.us> References: <3673.1188602373@sss.pgh.pa.us> <5802.1188611110@sss.pgh.pa.us> <5916.1188611810@sss.pgh.pa.us> Content-Type: text/plain Date: Sun, 02 Sep 2007 22:06:40 +0100 Message-Id: <1188767200.4167.54.camel@ebony.site> Mime-Version: 1.0 X-Mailer: Evolution 2.8.2 Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200709/6 X-Sequence-Number: 26767 On Fri, 2007-08-31 at 21:56 -0400, Tom Lane wrote: > Jeff Frost writes: > > That all seems reasonable enough. Is it in the docs somewhere? I > > didn't find anything like this mentioned. If not, could we get it > > added as a note? > > Yeah, it hadn't occurred to anyone to specify this, because we just > thought of recovery_command as fetching from a static archive. > We clearly need to document the expected semantics better. > > I'm wondering whether we should discourage people from putting > side-effects into the recovery_command, period. You already found out > that recovery can ask for the same file more than once, but what if it > never asks for a particular file at all? I'm not sure that can happen, > just playing devil's advocate. Yes, side effects are bad, in general. > Simon, did you see this thread? What do you think? The two main things we want are: 1. avoid doing a COPY from the archive, to improve performance 2. automatically clear down the archive when running in continuous recovery mode At first thought, it seems easy to fix this by removing the double file request. That idea was in my original PITR patch, but it was removed and sensibly so. The subtle point is that if you remove files from the archive too early then you may prevent the recovery from being restarted. So "mv" should never be used to avoid the copy operation, because it implements (1) but not (2). I've submitted a patch that sends the recovery_command a new %r value which is the last file needed to restart the recovery. All files prior to that file can be removed from the archive. That is then used by pg_standby to maintain the archive. We discussed the patch as being for 8.3, but the patch is currently in the 8.4 queue. That patch solves this issue, so I'd ask that we review that now. (pgsql-patches, 8 April). That includes initial docs on pg_standby, which does implement file management correctly. There *is* a bug in PITR which I fixed recently. (pgsql-patches, 8 June). That one really needs to be in 8.3 - more review work, sorry. Whatever we do, there's more docs coming on this for 8.3, pg_standby, pg_compresslog and a few other points. Give me a week or so please. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com