Date: Mon, 20 May 2019 22:55:53 +0200
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Bimal <internetuser2008@yahoo.com>
Cc: Greg Clough <Greg.Clough@ihsmarkit.com>,
	Mariel Cherkassky <mariel.cherkassky@gmail.com>,
	"pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Subject: Re: Trying to handle db corruption 9.6
Message-ID: <20190520205553.f5fst3zleabczf7t@development>
References: 
 <CA+t6e1=AHht9vqazjB8s15KH2Y7Vs1br3AVZpU5v3fJZ-Eg9Pw@mail.gmail.com>
 <CAHogYcXfPf6gJbXQgkOcRfjL-pJdxApvkzF3YowNz5r_5KMG7w@mail.gmail.com>
 <CA+t6e1=uKKccO4cGPdXgM4K3+fa1sY9-PE+w8V7QAHS_P1ot-g@mail.gmail.com>
 <DM6PR12MB27969B7AF5108691B55A00DEEC060@DM6PR12MB2796.namprd12.prod.outlook.com>
 <CA+t6e1=j8hwY-LCgN461-bCNB4ODxG6LNTHD9=uHko7ZAHy-fA@mail.gmail.com>
 <1909968146.4951657.1558369245203@mail.yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
In-Reply-To: <1909968146.4951657.1558369245203@mail.yahoo.com>
User-Agent: NeoMutt/20180716-1444-295967
Precedence: bulk

On Mon, May 20, 2019 at 04:20:45PM +0000, Bimal wrote:
>   I had ran into same issue about year back, luckily I had standby to
>   quickly promote.  But, I wish there was better a documentation on how to
>   handle WAL log fill up and resetting them.  

pg_resetxlog is not a tool to deal with "WAL fill up". It's a last
resort option to deal with corrupted WAL, and can easily make matters
worse when used without due consideration. That seems to be the case
here, unfortunately.

On a properly behaving system, running out of disk space for pg_xlog
results in database shutdown. If you also get corrupted WAL, you have
bigger problems, I'm afraid.

Also, data corruption issues are one-off events, mostly unique. That
makes it rather difficult (~impossible) to write docs about recovering
from them. And it's why there are no magic tools.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services