From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Mariel Cherkassky <mariel.cherkassky@gmail.com>
Cc: Bimal <internetuser2008@yahoo.com>
Cc: Greg Clough <Greg.Clough@ihsmarkit.com>
Cc: PostgreSQL mailing lists <pgsql-performance@postgresql.org>
Subject: Re: Trying to handle db corruption 9.6
Date: Mon, 20 May 2019 23:04:06 +0200
Message-ID: <20190520210406.5paaijqd5no6imez@development> (raw)
In-Reply-To: <CA+t6e1=+Wmw8WwrMTQR0umR3ktkGiVfFUho8xeSni2Jjm4wKtg@mail.gmail.com>
References: <CA+t6e1=AHht9vqazjB8s15KH2Y7Vs1br3AVZpU5v3fJZ-Eg9Pw@mail.gmail.com>
<CAHogYcXfPf6gJbXQgkOcRfjL-pJdxApvkzF3YowNz5r_5KMG7w@mail.gmail.com>
<CA+t6e1=uKKccO4cGPdXgM4K3+fa1sY9-PE+w8V7QAHS_P1ot-g@mail.gmail.com>
<DM6PR12MB27969B7AF5108691B55A00DEEC060@DM6PR12MB2796.namprd12.prod.outlook.com>
<CA+t6e1=j8hwY-LCgN461-bCNB4ODxG6LNTHD9=uHko7ZAHy-fA@mail.gmail.com>
<1909968146.4951657.1558369245203@mail.yahoo.com>
<CA+t6e1=+Wmw8WwrMTQR0umR3ktkGiVfFUho8xeSni2Jjm4wKtg@mail.gmail.com>
On Mon, May 20, 2019 at 08:20:33PM +0300, Mariel Cherkassky wrote:
> Hey Greg,
> Basically my backup was made after the first pg_resetxlog so I was wrong.
Bummer.
> However, the customer had a secondary machine that wasn't synced for a
> month. I have all the walls since the moment the secondary went out of
> sync. Once I started it I hoped that it will start recover the wals and
> fill the gap. However I got an error in the secondary :
> 2019-05-20 10:11:28 PDT 19021 LOG: entering standby mode
> 2019-05-20 10:11:28 PDT 19021 LOG: invalid primary checkpoint record
> 2019-05-20 10:11:28 PDT 19021 LOG: invalid secondary checkpoint link in
> control file
> 2019-05-20 10:11:28 PDT 19021 PANIC: could not locate a valid
> checkpoint record
> 2019-05-20 10:11:28 PDT 19018 LOG: startup process (PID 19021) was
> terminated by signal 6: Aborted
> 2019-05-20 10:11:28 PDT 19018 LOG: aborting startup due to startup
> process failure
> 2019-05-20 10:11:28 PDT 19018 LOG: database system is shut down.
> I checked my secondary archive dir and pg_xlog dir and
> it seems that the restore command doesnt work. My restore_command:
> restore_command = 'rsync -avzhe ssh
> postgres@x.x.x.x:/var/lib/pgsql/archive/%f /var/lib/pgsql/archive/%f ;
> gunzip < /var/lib/pgsql/archive/%f > %p'
> archive_cleanup_command = '/usr/pgsql-9.6/bin/pg_archivecleanup
> /var/lib/pgsql/archive %r'
Well, when you say it does not work, why do you think so? Does it print
some error, or what? Does it even get executed? It does not seem to be
the case, judging by the log (there's no archive_command message).
How was the "secondary machine" created? You said you have all the WAL
since then - how do you know that?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, mariel.cherkassky@gmail.com, internetuser2008@yahoo.com, Greg.Clough@ihsmarkit.com
Subject: Re: Trying to handle db corruption 9.6
In-Reply-To: <20190520210406.5paaijqd5no6imez@development>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox