Date: Thu, 13 May 2010 10:25:46 -0400
From: Ray Stell <stellr@cns.vt.edu>
To: Devrim G?ND?Z <devrim@gunduz.org>
Cc: pgsql-admin@postgresql.org
Subject: Re: upgrade proc with pitr standby
Message-ID: <20100513142546.GH22313@cns.vt.edu>
References: <20100511131617.GA17277@cns.vt.edu>
	<1273584558.5575.43.camel@hp-laptop2.gunduz.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1273584558.5575.43.camel@hp-laptop2.gunduz.org>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Tue, May 11, 2010 at 04:29:18PM +0300, Devrim G?ND?Z wrote:
> On Tue, 2010-05-11 at 09:16 -0400, Ray Stell wrote:
> > What are the steps for upgrade of a primary/PITR standby pair to the 
> > latest 8.3 patchset, 8.3.6-8.3.10?  I don't see this in the docs. 
> > Should there be something added to 15.4. Upgrading?   
> > 
> > My guess is:
> > 1. turn off the standby config in primary
> > 2. upgrade primary
> > 3. rebuild standby 
> 
> You need to upgrade standby first, restart it. Then, upgrade master and
> restart it. No need to turn off standby config.

do you know why might be going south?

the standby is applying WAL:
----------------------------
,30357,,2010-05-12 17:04:33.927 EDT,4beb17d2.7695,5,2010-05-12 17:04:18 EDT,0, LOG:  automatic recovery in progress
,30357,,2010-05-12 17:04:34.534 EDT,4beb17d2.7695,6,2010-05-12 17:04:18 EDT,0, LOG:  redo starts at 0/567D982C
,30357,,2010-05-12 17:04:34.630 EDT,4beb17d2.7695,7,2010-05-12 17:04:18 EDT,0, LOG:  restored log file "000000020000000000000057" from archive
,30357,,2010-05-12 17:04:35.315 EDT,4beb17d2.7695,8,2010-05-12 17:04:18 EDT,0, LOG:  restored log file "000000020000000000000058" from archive
...

,30357,,2010-05-12 17:05:21.555 EDT,4beb17d2.7695,23,2010-05-12 17:04:18 EDT,0, LOG:  restored log file "000000020000000000000067" from archive
,30357,,2010-05-12 17:05:22.408 EDT,4beb17d2.7695,24,2010-05-12 17:04:18 EDT,0, LOG:  restored log file "000000020000000000000068" from archive


shutdown standby:
-----------------
,30355,,2010-05-13 08:59:49.547 EDT,4beb17d2.7693,1,2010-05-12 17:04:18 EDT,0, LOG:  received fast shutdown request
,30357,,2010-05-13 08:59:49.547 EDT,4beb17d2.7695,25,2010-05-12 17:04:18 EDT,0, FATAL:  could not restore file "000000020000000000000069" from archive: return code 15
,30355,,2010-05-13 08:59:49.551 EDT,4beb17d2.7693,2,2010-05-12 17:04:18 EDT,0, LOG:  startup process (PID 30357) exited with exit code 1
,30355,,2010-05-13 08:59:49.551 EDT,4beb17d2.7693,3,2010-05-12 17:04:18 EDT,0, LOG:  aborting startup due to startup process failure


flip to 8.3.10 and restart standby:
-----------------------------------
,3440,,2010-05-13 09:06:35.720 EDT,4bebf95b.d70,1,2010-05-13 09:06:35 EDT,0, LOG:  database system was interrupted while in recovery at log time 2010-05-12 17:01:05 EDT
,3440,,2010-05-13 09:06:35.720 EDT,4bebf95b.d70,2,2010-05-13 09:06:35 EDT,0, HINT:  If this has occurred more than once some data might be corrupted and you might need to choose an earlier recovery target.
,3440,,2010-05-13 09:06:35.720 EDT,4bebf95b.d70,3,2010-05-13 09:06:35 EDT,0, LOG:  starting archive recovery
,3440,,2010-05-13 09:06:35.732 EDT,4bebf95b.d70,4,2010-05-13 09:06:35 EDT,0, LOG:  restore_command = '/usr/local/pgsql/bin/pg_standby  /data/pgsql/wals/alerts_oamp %f %p %r'
sh: /usr/local/pgsql/bin/pg_standby: No such file or directory
,3440,,2010-05-13 09:06:35.734 EDT,4bebf95b.d70,5,2010-05-13 09:06:35 EDT,0, FATAL:  could not restore file "00000002.history" from archive: return code 32512
,3438,,2010-05-13 09:06:35.735 EDT,4bebf95b.d6e,1,2010-05-13 09:06:35 EDT,0, LOG:  startup process (PID 3440) exited with exit code 1
,3438,,2010-05-13 09:06:35.735 EDT,4bebf95b.d6e,2,2010-05-13 09:06:35 EDT,0, LOG:  aborting startup due to startup process failure