Date: Tue, 24 Mar 2020 13:40:16 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Andres Freund <andres@anarazel.de>
Cc: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>,
	Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>,
	sawada.mshk@gmail.com, tgl@sss.pgh.pa.us, alvherre@2ndquadrant.com,
	sk@zsrv.org, nasbyj@amazon.com, robertmhaas@gmail.com,
	pgsql-hackers@lists.postgresql.org
Subject: Re: pgsql: Improve autovacuum logging for aggressive and
 anti-wraparound ru
Message-ID: <20200324044016.GD1836@paquier.xyz>
References: 
 <CAD21AoBR+y31ps+=G6_fxxVd_o1itTnaK19sDAEhAP_RyVT2AA@mail.gmail.com>
 <20181005031603.GF1629@paquier.xyz>
 <20181005063504.GB14664@paquier.xyz>
 <20181009.181536.142257785.horiguchi.kyotaro@lab.ntt.co.jp>
 <920b7f84-407c-005c-4b7d-e8266a17d22b@2ndQuadrant.com>
 <20190309011537.GD9266@paquier.xyz>
 <20190329115138.GD1954@paquier.xyz>
 <20200324014150.2nyuygbbejzaybip@alap3.anarazel.de>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="84ND8YJRMFlzkrP4"
Content-Disposition: inline
In-Reply-To: <20200324014150.2nyuygbbejzaybip@alap3.anarazel.de>
Precedence: bulk


--84ND8YJRMFlzkrP4
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Mon, Mar 23, 2020 at 06:41:50PM -0700, Andres Freund wrote:
> Which valid scenario can lead to this? Neither the comment, nor commit
> message explain it.

The commit message mentions that concurrent autovacuum jobs can lead
to the creation of non-aggressive and anti-wraparound jobs, which have
no sense because an aggressive and anti-wraparound job was already
done in parallel with a different worker, and that this was possible
because of inconsistent relcache lookups across concurrent jobs.  This
was mentioned upthread.

> Unless you're thinking of scenarios where autovacuum
> and manual vacuum are mixed, I don't really see valid reasons? Normally
> autovacuum's locking + the table_recheck_autovac() check should prevent
> problematic scenarios.
>
> I do see a few scenarios that can trigger this - but they all more or
> less are bugs.

Hmm.  OK.

> It doesn't strike me as a good idea to work around such bugs by silently
> neutering heap_vacuum_rel(). The likelihood of that temporarily covering
> up more severe problems seems significant - they're likely to then later
> bite you with a cluster shutdown.

Saying that, I have been thinking about this one for a couple of days
now and it seems to me that this is a factor contributing to what we
are seeing in [1], and I agree that this is just an incorrect approach
that makes easier to trigger the real underlying issues, while
table_recheck_autovac() ought to be the only code path doing the skip
job.  Note that I have failed to reproduce the behavior of the other
thread though, always finishing with a non-aggressive anti-wraparound
skipped because of an aggressive and anti-wraparound job happened just
before in parallel, and autovacuum was always able to continue
triggering new jobs, keeping the relfrozenxid age at bay.

So I would like to first revert that part, to have a cleaner state to
work on the underlying issues.  A pure revert means also adding back
the log message for non-aggressive and anti-wraparound jobs that
should never exist, which should be replaced by an assertion once all
the holes are fixed.  What do you think?

[1]: https://www.postgresql.org/message-id/CAE39h23RTX1jkYjWc5tccv34HwwraizaCUxOmdQdPM+Zt5-2Qg@mail.gmail.com
--
Michael

--84ND8YJRMFlzkrP4
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAl55jzAACgkQnvQgOdby
QH2YtQ/9ESsLGgjFV4QH6vJYTkQL896sTAqap+tL6yFFVNfx1yTZKzuR1Fa/AuQm
sZEkFDcLhvH5sak8lBxLzeDXabESc/vhTwUZ8heoAbHvrRffana+MWqDgm85edVR
P/5MnvtYSgZbMhH1cFk8IirbkcdGSmSfhyamYc45YLEHwFoko0fTCGj2weEa5/Df
C54HzA4/tukzMpnNb8IfW/6lgkU6b57YtSSMY2T75ATOza+MJ1zoG7Q/NKQnZEgS
sieN8ofY5hemNraEpTdiO5+Gmh46Ccw5+Q0X/cXIu4VZ1X/tCK5Wty9xMc00Wun3
kJ9JIabgv+mhTyWQTOHn8rBm60Of/oVefyeXeGAL3uKV1hQJaHlzPOt+LxhOWtXk
1d1plqgg2+rl9UlWB+t9VCCet64FkgfDJ7j3miMPDzZ3Hq6Km2BmYginB8X+tKuE
OMkxNkZgKqeSCM+USx8+VTwiN87LT/O64mc3xCGaHpBK3sdRxNrLweFqRb9HAMuB
Q+S+clSbf/0p67pFEqyFu1pGSYb9fnOjAKK2L0dY9O/CBokjw010uU/i1D5KRl6o
5Lo0OrPy/An5TEcSCQftDH7LfNfSuwq/0M7T6k89qwljYg15DWeeuFramcMn5tRp
wqUEoTTzaktf9pff9rdSsNnIbkqfsgt+C0NlDTNvHzbG73HUY2U=
=vO24
-----END PGP SIGNATURE-----

--84ND8YJRMFlzkrP4--