From: Nasby, Jim <nasbyj@amazon.com>
To: Andres Freund <andres@anarazel.de>
Cc: Sergei Kornilov <sk@zsrv.org>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru
Date: Mon, 24 Sep 2018 18:25:46 +0000
Message-ID: <0A11E30F-DDDE-422F-B22A-3F6E3AA7A0A2@amazon.com> (raw)
In-Reply-To: <20180921174347.wvszsn7uk56zvz7o@alap3.anarazel.de>
References: <E1g0aGi-0000uN-Sq@gemulon.postgresql.org>
<20180914153554.562muwr3uwujno75@alvherre.pgsql>
<CA+Tgmoa7JJG0g2rCR=m8wXzExZDBpHPo04a5FyV-EotknVDONg@mail.gmail.com>
<7924791537551496@myt5-c56023d17c6b.qloud-c.yandex.net>
<20180921174347.wvszsn7uk56zvz7o@alap3.anarazel.de>
> On Sep 21, 2018, at 12:43 PM, Andres Freund <andres@anarazel.de> wrote:
>
>> But as far i can see it is possible have aggressive non-wraparound vacuum. One important difference - regular and aggressive regular can be canceled by backend,.wraparound autovacuum can not. (by checking PROC_VACUUM_FOR_WRAPAROUND in src/backend/storage/lmgr/proc.c )
>
> Yes, without checking the code, they should be different. Aggressive is
> controlled by vacuum_freeze_table_age whereas anti-wrap is controlled by
> autovacuum_freeze_max_age (but also implies aggressive).
Right, except that by the time you get into the vacuum code itself nothing should really care about that difference. AFAICT, the only thing is_wraparound is being used for is to set MyPgXact->vacuumFlags |= PROC_VACUUM_FOR_WRAPAROUND, which prevents the deadlock detector from killing an autovac process that’s trying to prevent a wraparound. I think it’d be clearer to remove is_wraparound and move the check from vacuum_rel() into lazy_vacuum_rel() (which is where the limits for HeapTupleSatisfiesVacuum get determined). Something like the attached.
Attachments:
[application/octet-stream] anti_wrap.patch (5.5K, ../0A11E30F-DDDE-422F-B22A-3F6E3AA7A0A2@amazon.com/2-anti_wrap.patch)
download | inline diff:diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.cindex f166509734..32bb009953 100644--- a/src/backend/commands/vacuum.c+++ b/src/backend/commands/vacuum.c@@ -131,9 +131,6 @@ ExecVacuum(VacuumStmt *vacstmt, bool isTopLevel)
params.multixact_freeze_table_age = -1;
}
- /* user-invoked vacuum is never "for wraparound" */- params.is_wraparound = false;-
/* user-invoked vacuum never uses this parameter */
params.log_min_duration = -1;
@@ -1437,8 +1434,6 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
*/
LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
MyPgXact->vacuumFlags |= PROC_IN_VACUUM;
- if (params->is_wraparound)- MyPgXact->vacuumFlags |= PROC_VACUUM_FOR_WRAPAROUND;
LWLockRelease(ProcArrayLock);
}
diff --git a/src/backend/commands/vacuumlazy.c b/src/backend/commands/vacuumlazy.cindex 8996d366e9..2514405c7d 100644--- a/src/backend/commands/vacuumlazy.c+++ b/src/backend/commands/vacuumlazy.c@@ -199,6 +199,7 @@ lazy_vacuum_rel(Relation onerel, int options, VacuumParams *params,
int usecs;
double read_rate,
write_rate;
+ bool anti_wraparound; /* is this vacuum to prevent wraparound */
bool aggressive; /* should we scan all unfrozen pages? */
bool scanned_all_unfrozen; /* actually scanned all such pages? */
TransactionId xidFullScanLimit;
@@ -242,12 +243,24 @@ lazy_vacuum_rel(Relation onerel, int options, VacuumParams *params,
* table's minimum MultiXactId is older than or equal to the requested
* mxid full-table scan limit; or if DISABLE_PAGE_SKIPPING was specified.
*/
- aggressive = TransactionIdPrecedesOrEquals(onerel->rd_rel->relfrozenxid,+ anti_wraparound = TransactionIdPrecedesOrEquals(onerel->rd_rel->relfrozenxid,
xidFullScanLimit);
- aggressive |= MultiXactIdPrecedesOrEquals(onerel->rd_rel->relminmxid,+ anti_wraparound |= MultiXactIdPrecedesOrEquals(onerel->rd_rel->relminmxid,
mxactFullScanLimit);
if (options & VACOPT_DISABLE_PAGE_SKIPPING)
aggressive = true;
+ else+ aggressive = anti_wraparound;++ /*+ * If this is an anti-wraparound vacuum, prevent it from being killed by+ * the deadlock detector.+ */+ if (anti_wraparound && IsAutoVacuumWorkerProcess()) {+ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);+ MyPgXact->vacuumFlags |= PROC_VACUUM_FOR_WRAPAROUND;+ LWLockRelease(ProcArrayLock);+ }
vacrelstats = (LVRelStats *) palloc0(sizeof(LVRelStats));
@@ -261,6 +274,23 @@ lazy_vacuum_rel(Relation onerel, int options, VacuumParams *params,
vac_open_indexes(onerel, RowExclusiveLock, &nindexes, &Irel);
vacrelstats->hasindex = (nindexes > 0);
+ /* Report status. Autovac status is handled below. */+ if (!IsAutoVacuumWorkerProcess()) {+ char *msgfmt;++ if (anti_wraparound)+ msgfmt = _("vacuuming to prevent wraparound of table \"%s.%s\"");+ else if (aggressive)+ msgfmt = _("vacuuming \"%s.%s\" without page skipping");+ else+ msgfmt = _("vacuuming \"%s.%s\"");++ ereport(elevel,+ (errmsg(msgfmt,+ get_namespace_name(RelationGetNamespace(onerel)),+ RelationGetRelationName(onerel))));+ }+
/* Do the vacuuming */
lazy_scan_heap(onerel, options, vacrelstats, Irel, nindexes, aggressive);
@@ -374,20 +404,11 @@ lazy_vacuum_rel(Relation onerel, int options, VacuumParams *params,
* emitting individual parts of the message when not applicable.
*/
initStringInfo(&buf);
- if (params->is_wraparound)- {- if (aggressive)- msgfmt = _("automatic aggressive vacuum to prevent wraparound of table \"%s.%s.%s\": index scans: %d\n");- else- msgfmt = _("automatic vacuum to prevent wraparound of table \"%s.%s.%s\": index scans: %d\n");- }+ if (anti_wraparound)+ msgfmt = _("automatic vacuum to prevent wraparound of table \"%s.%s.%s\": index scans: %d\n");
else
- {- if (aggressive)- msgfmt = _("automatic aggressive vacuum of table \"%s.%s.%s\": index scans: %d\n");- else- msgfmt = _("automatic vacuum of table \"%s.%s.%s\": index scans: %d\n");- }+ /* An autovacuum can't be just aggressive; it's either anti-wraparound or normal. */+ msgfmt = _("automatic vacuum of table \"%s.%s.%s\": index scans: %d\n");
appendStringInfo(&buf, msgfmt,
get_database_name(MyDatabaseId),
get_namespace_name(RelationGetNamespace(onerel)),
@@ -505,16 +526,6 @@ lazy_scan_heap(Relation onerel, int options, LVRelStats *vacrelstats,
pg_rusage_init(&ru0);
relname = RelationGetRelationName(onerel);
- if (aggressive)- ereport(elevel,- (errmsg("aggressively vacuuming \"%s.%s\"",- get_namespace_name(RelationGetNamespace(onerel)),- relname)));- else- ereport(elevel,- (errmsg("vacuuming \"%s.%s\"",- get_namespace_name(RelationGetNamespace(onerel)),- relname)));
empty_pages = vacuumed_pages = 0;
next_fsm_block_to_vacuum = (BlockNumber) 0;
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.hindex 5af96fdc8a..e07a90f164 100644--- a/src/include/commands/vacuum.h+++ b/src/include/commands/vacuum.h@@ -142,7 +142,6 @@ typedef struct VacuumParams
* use default */
int multixact_freeze_table_age; /* multixact age at which to scan
* whole table */
- bool is_wraparound; /* force a for-wraparound vacuum */
int log_min_duration; /* minimum execution threshold in ms at
* which verbose logs are activated, -1
* to use default */
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: nasbyj@amazon.com, andres@anarazel.de, sk@zsrv.org, robertmhaas@gmail.com, alvherre@2ndquadrant.com, michael@paquier.xyz, pgsql-hackers@lists.postgresql.org
Subject: Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru
In-Reply-To: <0A11E30F-DDDE-422F-B22A-3F6E3AA7A0A2@amazon.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox