Date: Fri, 27 Mar 2020 20:23:03 +0100
From: Julien Rouhaud <rjuju123@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Justin King <kingpin867@gmail.com>, Andres Freund <andres@anarazel.de>,
	pgsql-general@lists.postgresql.org, kgrittn@gmail.com
Subject: Re: PG12 autovac issues
Message-ID: <20200327192303.GA20785@nol>
References: <20200323194725.GC52612@nol>
 <20200323200051.qzyqvnips7o3ai4g@alap3.anarazel.de>
 <20200324052606.GE1836@paquier.xyz>
 <20200324054039.ulrxbzo2ycy7pu5i@alap3.anarazel.de>
 <20200324061238.GG1836@paquier.xyz>
 <20200324194259.pidupqxtqnquk4wu@alap3.anarazel.de>
 <CAE39h23EoQpGeBN4XDg5MhZZFz_AMbnr7O+h-Q=LXvS1ZuoXyQ@mail.gmail.com>
 <20200326014108.GA2199@paquier.xyz>
 <CAE39h20kVBn+wHGBapOyoDUSAq=6vTyO4KuoX4sHQtZfdpVi+A@mail.gmail.com>
 <20200327050622.GB2133@paquier.xyz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200327050622.GB2133@paquier.xyz>
Precedence: bulk

On Fri, Mar 27, 2020 at 02:12:04PM +0900, Michael Paquier wrote:
> On Thu, Mar 26, 2020 at 09:46:47AM -0500, Justin King wrote:
> > Nope, it was just these tables that were looping over and over while
> > nothing else was getting autovac'd.  I'm happy to share the full log
> > if you'd like.
> 
> Thanks, that could help.  If that's very large, it could be a problem
> to send that to the lists, but you could send me directly a link to
> it and I'll try to extract more information for the lists.  While
> testing for reproducing the issue, I have noticed that basically one
> set of catalog tables happened to see this "skipping redundant" log. 
> And I am wondering if we have a match with the set of catalog tables
> looping.
> 
> > I did have to remove it from this state, but I can undo my workaround
> > and, undoubtedly, it'll end up back there.  Let me know if there's
> > something specific you'd like me to provide when it happens!
> 
> For now I think it's fine.  Note that Julien and I have an environment
> where the issue can be reproduced easily (it takes roughly 12 hours
> until the wraparound cutoffs are reached with the benchmark and
> settings used), and we are checking things using a patched instance
> with 2aa6e33 reverted.  I think that we are accumulating enough
> evidence that this change was not a good idea anyway thanks to the
> information you sent, so likely we'll finish first by a revert of
> 2aa6e33 from the master and REL_12_STABLE branches, before looking at
> the issues with the catalogs for those anti-wraparound and
> non-aggressive jobs (this looks like a relcache issue with the so-said
> catalogs).

FTR we reached the 200M transaxtion earlier, and I can see multiple logs of the
form "automatic vacuum to prevent wraparound", so non-aggressive antiwraparound
autovacuum, all on shared relations.

As those vacuum weren't skipped, autovacuum didn't get stuck in a loop on those
and continue its work normally.  This happened ~ 4h ago, didn't ocurred again
while the 200M threshold was reached again multiple time.