From: Michael Paquier <michael@paquier.xyz>
To: Andres Freund <andres@anarazel.de>
Cc: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: sawada.mshk@gmail.com
Cc: tgl@sss.pgh.pa.us
Cc: alvherre@2ndquadrant.com
Cc: sk@zsrv.org
Cc: nasbyj@amazon.com
Cc: robertmhaas@gmail.com
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru
Date: Tue, 24 Mar 2020 13:40:16 +0900
Message-ID: <20200324044016.GD1836@paquier.xyz> (raw)
In-Reply-To: <20200324014150.2nyuygbbejzaybip@alap3.anarazel.de>
References: <CAD21AoBR+y31ps+=G6_fxxVd_o1itTnaK19sDAEhAP_RyVT2AA@mail.gmail.com>
<20181005031603.GF1629@paquier.xyz>
<20181005063504.GB14664@paquier.xyz>
<20181009.181536.142257785.horiguchi.kyotaro@lab.ntt.co.jp>
<920b7f84-407c-005c-4b7d-e8266a17d22b@2ndQuadrant.com>
<20190309011537.GD9266@paquier.xyz>
<20190329115138.GD1954@paquier.xyz>
<20200324014150.2nyuygbbejzaybip@alap3.anarazel.de>
On Mon, Mar 23, 2020 at 06:41:50PM -0700, Andres Freund wrote:
> Which valid scenario can lead to this? Neither the comment, nor commit
> message explain it.
The commit message mentions that concurrent autovacuum jobs can lead
to the creation of non-aggressive and anti-wraparound jobs, which have
no sense because an aggressive and anti-wraparound job was already
done in parallel with a different worker, and that this was possible
because of inconsistent relcache lookups across concurrent jobs. This
was mentioned upthread.
> Unless you're thinking of scenarios where autovacuum
> and manual vacuum are mixed, I don't really see valid reasons? Normally
> autovacuum's locking + the table_recheck_autovac() check should prevent
> problematic scenarios.
>
> I do see a few scenarios that can trigger this - but they all more or
> less are bugs.
Hmm. OK.
> It doesn't strike me as a good idea to work around such bugs by silently
> neutering heap_vacuum_rel(). The likelihood of that temporarily covering
> up more severe problems seems significant - they're likely to then later
> bite you with a cluster shutdown.
Saying that, I have been thinking about this one for a couple of days
now and it seems to me that this is a factor contributing to what we
are seeing in [1], and I agree that this is just an incorrect approach
that makes easier to trigger the real underlying issues, while
table_recheck_autovac() ought to be the only code path doing the skip
job. Note that I have failed to reproduce the behavior of the other
thread though, always finishing with a non-aggressive anti-wraparound
skipped because of an aggressive and anti-wraparound job happened just
before in parallel, and autovacuum was always able to continue
triggering new jobs, keeping the relfrozenxid age at bay.
So I would like to first revert that part, to have a cleaner state to
work on the underlying issues. A pure revert means also adding back
the log message for non-aggressive and anti-wraparound jobs that
should never exist, which should be replaced by an assertion once all
the holes are fixed. What do you think?
[1]: https://www.postgresql.org/message-id/CAE39h23RTX1jkYjWc5tccv34HwwraizaCUxOmdQdPM+Zt5-2Qg@mail.gmail...
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../20200324044016.GD1836@paquier.xyz/2-signature.asc)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: michael@paquier.xyz, andres@anarazel.de, andrew.dunstan@2ndquadrant.com, horiguchi.kyotaro@lab.ntt.co.jp, sawada.mshk@gmail.com, tgl@sss.pgh.pa.us, alvherre@2ndquadrant.com, sk@zsrv.org, nasbyj@amazon.com, robertmhaas@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru
In-Reply-To: <20200324044016.GD1836@paquier.xyz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox