agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedanother autovacuum scheduling thread
150+ messages / 16 participants
[nested] [flat]
* another autovacuum scheduling thread
@ 2025-10-08 15:18 Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 17:37 ` Re: another autovacuum scheduling thread Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 150+ messages in thread
From: Nathan Bossart @ 2025-10-08 15:18 UTC (permalink / raw)
To: pgsql-hackers
/me dons flame-proof suit
My goal with this thread is to produce some incremental autovacuum
scheduling improvements for v19, but realistically speaking, I know that
it's a bit of a long-shot. There have been many discussions over the
years, and I've read through a few of them [0] [1] [2] [3] [4], but there
are certainly others I haven't found. Since this seems to be a contentious
topic, I figured I'd start small to see if we can get _something_
committed.
While I am by no means wedded to a specific idea, my current concrete
proposal (proof-of-concept patch attached) is to start by ordering the
tables a worker will process by (M)XID age. Here are the reasons:
* We do some amount of prioritization of databases at risk of wraparound at
database level, per the following comment from autovacuum.c:
* Choose a database to connect to. We pick the database that was least
* recently auto-vacuumed, or one that needs vacuuming to prevent Xid
* wraparound-related data loss. If any db at risk of Xid wraparound is
* found, we pick the one with oldest datfrozenxid, independently of
* autovacuum times; similarly we pick the one with the oldest datminmxid
* if any is in MultiXactId wraparound. Note that those in Xid wraparound
* danger are given more priority than those in multi wraparound danger.
However, we do no such prioritization of the tables within a database. In
fact, the ordering of the tables is effectively random. IMHO this gives us
quite a bit of wiggle room to experiment; since we are processing tables in
no specific order today, changing the order to something vacuuming-related
seems more likely to help than it is to harm.
* Prioritizing tables based on their (M)XID age might help avoid more
aggressive vacuums, not to mention wraparound. Of course, there are
scenarios where this doesn't work. For example, the age of a table may
have changed greatly between the time we recorded it and the time we
process it. Or maybe there is another table in a different database that
is more important from a wraparound perspective. We could complicate the
patch to try to handle some of these things, but I maintain that even some
basic, incremental scheduling improvements would be better than the status
quo. And we can always change it further in the future to handle these
problems and to consider other things like bloat.
The attached patch works by storing the maximum of the XID age and the MXID
age in the list with the OIDs and sorting it prior to processing.
Thoughts?
[0] https://postgr.es/m/CA%2BTgmoafJPjB3WVqB3FrGWUU4NLRc3VHx8GXzLL-JM%2B%2BJPwK%2BQ%40mail.gmail.com
[1] https://postgr.es/m/CAEG8a3%2B3fwQbgzak%2Bh3Q7Bp%3DvK_aWhw1X7w7g5RCgEW9ufdvtA%40mail.gmail.com
[2] https://postgr.es/m/CAD21AoBUaSRBypA6pd9ZD%3DU-2TJCHtbyZRmrS91Nq0eVQ0B3BA%40mail.gmail.com
[3] https://postgr.es/m/CA%2BTgmobT3m%3D%2BdU5HF3VGVqiZ2O%2Bv6P5wN1Gj%2BPrq%2Bhj7dAm9AQ%40mail.gmail.com
[4] https://postgr.es/m/20130124215715.GE4528%40alvh.no-ip.org
--
nathan
From 7930f6ac213b9145489c4e9253872c8bb4fb5855 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 8 Oct 2025 10:12:14 -0500
Subject: [PATCH v1 1/1] autovacuum: order tables by (m)xid age
---
src/backend/postmaster/autovacuum.c | 47 ++++++++++++++++++++++++-----
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 41 insertions(+), 7 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index fb5d3b27224..aeef28abd67 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -309,6 +309,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ uint32 age;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -1888,6 +1894,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return pg_cmp_u32(t2->age, t1->age);
+}
+
/*
* Process a database table-by-table
*
@@ -1901,7 +1916,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2055,9 +2070,17 @@ do_autovacuum(void)
effective_multixact_freeze_max_age,
&dovacuum, &doanalyze, &wraparound);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->age = recentXid - classForm->relfrozenxid;
+ table->age = Max(table->age, recentMulti - classForm->relminmxid);
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2149,7 +2172,15 @@ do_autovacuum(void)
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->age = recentXid - classForm->relfrozenxid;
+ table->age = Max(table->age, recentMulti - classForm->relminmxid);
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2273,6 +2304,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2301,9 +2334,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2534,7 +2567,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 02b5b041c45..1abfc338760 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3004,6 +3004,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v1-0001-autovacuum-order-tables-by-m-xid-age.patch (3.9K, ../../aOaAuXREwnPZVISO@nathan/2-v1-0001-autovacuum-order-tables-by-m-xid-age.patch)
download | inline diff:
From 7930f6ac213b9145489c4e9253872c8bb4fb5855 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 8 Oct 2025 10:12:14 -0500
Subject: [PATCH v1 1/1] autovacuum: order tables by (m)xid age
---
src/backend/postmaster/autovacuum.c | 47 ++++++++++++++++++++++++-----
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 41 insertions(+), 7 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index fb5d3b27224..aeef28abd67 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -309,6 +309,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ uint32 age;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -1888,6 +1894,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return pg_cmp_u32(t2->age, t1->age);
+}
+
/*
* Process a database table-by-table
*
@@ -1901,7 +1916,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2055,9 +2070,17 @@ do_autovacuum(void)
effective_multixact_freeze_max_age,
&dovacuum, &doanalyze, &wraparound);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->age = recentXid - classForm->relfrozenxid;
+ table->age = Max(table->age, recentMulti - classForm->relminmxid);
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2149,7 +2172,15 @@ do_autovacuum(void)
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->age = recentXid - classForm->relfrozenxid;
+ table->age = Max(table->age, recentMulti - classForm->relminmxid);
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2273,6 +2304,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2301,9 +2334,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2534,7 +2567,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 02b5b041c45..1abfc338760 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3004,6 +3004,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-08 17:06 ` Sami Imseih <samimseih@gmail.com>
2025-10-08 17:20 ` Re: another autovacuum scheduling thread Álvaro Herrera <alvherre@kurilemu.de>
2025-10-08 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
1 sibling, 3 replies; 150+ messages in thread
From: Sami Imseih @ 2025-10-08 17:06 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers
Thanks for raising this topic! I agree that autovacuum scheduling
could be improved.
> * Prioritizing tables based on their (M)XID age might help avoid more
> aggressive vacuums, not to mention wraparound. Of course, there are
> scenarios where this doesn't work. For example, the age of a table may
> have changed greatly between the time we recorded it and the time we
> process it. Or maybe there is another table in a different database that
> is more important from a wraparound perspective. We could complicate the
> patch to try to handle some of these things, but I maintain that even some
> basic, incremental scheduling improvements would be better than the status
> quo. And we can always change it further in the future to handle these
> problems and to consider other things like bloat.
One risk I see with this approach is that we will end up autovacuuming
tables that also take the longest time to complete, which could cause
smaller, quick-to-process tables to be neglected.
It’s not always the case that the oldest tables in terms of (M)XID age
are also the most expensive to vacuum, but that is often more true
than not.
Not saying that the current approach, which is as you mention is
random, is any better, however this approach will likely increase
the behavior of large tables saturating workers.
But I also do see the merit of this approach when we know we are
in failsafe territory, because I would want my oldest aged tables to be
a/v'd first.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-08 17:20 ` Álvaro Herrera <alvherre@kurilemu.de>
2 siblings, 0 replies; 150+ messages in thread
From: Álvaro Herrera @ 2025-10-08 17:20 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On 2025-Oct-08, Sami Imseih wrote:
> One risk I see with this approach is that we will end up autovacuuming
> tables that also take the longest time to complete, which could cause
> smaller, quick-to-process tables to be neglected.
Perhaps we can have autovacuum workers decide on a mode to use at
startup (or launcher decides for them), and use different prioritization
heuristics depending on the mode. For instance if we're past max freeze
age for any tables then we know we have to first vacuum tables with
higher MXID ages regardless of size considerations, but if there's at
least one worker in that mode then we use the mode where smaller
high-churn tables go first.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"No nos atrevemos a muchas cosas porque son difíciles,
pero son difíciles porque no nos atrevemos a hacerlas" (Séneca)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-08 17:47 ` Sami Imseih <samimseih@gmail.com>
2 siblings, 0 replies; 150+ messages in thread
From: Sami Imseih @ 2025-10-08 17:47 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers
> Not saying that the current approach, which is as you mention is
> random, is any better, however this approach will likely increase
> the behavior of large tables saturating workers.
Maybe it will be good to allocate some workers to the oldest tables
and workers based on some random list? This could balance things
out between the oldest (large) tables and everything else to avoid
this problem.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-08 23:40 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2 siblings, 1 reply; 150+ messages in thread
From: Jeremy Schneider @ 2025-10-08 23:40 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Wed, 8 Oct 2025 12:06:29 -0500
Sami Imseih <samimseih@gmail.com> wrote:
>
> One risk I see with this approach is that we will end up autovacuuming
> tables that also take the longest time to complete, which could cause
> smaller, quick-to-process tables to be neglected.
>
> It’s not always the case that the oldest tables in terms of (M)XID age
> are also the most expensive to vacuum, but that is often more true
> than not.
I think an approach of doing largest objects first actually might work
really well for balancing work amongst autovacuum workers. Many years
ago I designed a system to backup many databases with a pool of workers
and used this same simple & naive algorithm of just reverse sorting on
db size, and it worked remarkably well. If you have one big thing then
you probably want someone to get started on that first. As long as
there's a pool of workers available, as you work through the queue, you
can actually end up with pretty optimal use of all the workers.
-Jeremy
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
@ 2025-10-08 23:59 ` David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-08 23:59 UTC (permalink / raw)
To: Jeremy Schneider <schneider@ardentperf.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Thu, 9 Oct 2025 at 12:41, Jeremy Schneider <schneider@ardentperf.com> wrote:
> I think an approach of doing largest objects first actually might work
> really well for balancing work amongst autovacuum workers. Many years
> ago I designed a system to backup many databases with a pool of workers
> and used this same simple & naive algorithm of just reverse sorting on
> db size, and it worked remarkably well. If you have one big thing then
> you probably want someone to get started on that first. As long as
> there's a pool of workers available, as you work through the queue, you
> can actually end up with pretty optimal use of all the workers.
I believe that is methodology for processing work applies much better
in scenarios where there's no new work continually arriving and
there's no adverse effects from giving a lower priority to certain
portions of the work. I don't think you can apply that so easily to
autovacuum as there are scenarios where the work can pile up faster
than it can be handled. Also, smaller tables can bloat in terms of
growth proportional to the original table size much more quickly than
larger tables and that could have huge consequences for queries to
small tables which are not indexed sufficiently to handle being
becoming bloated and large.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-09 00:27 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 00:30 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Jeremy Schneider @ 2025-10-09 00:27 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Thu, 9 Oct 2025 12:59:23 +1300
David Rowley <dgrowleyml@gmail.com> wrote:
> I believe that is methodology for processing work applies much better
> in scenarios where there's no new work continually arriving and
> there's no adverse effects from giving a lower priority to certain
> portions of the work. I don't think you can apply that so easily to
> autovacuum as there are scenarios where the work can pile up faster
> than it can be handled. Also, smaller tables can bloat in terms of
> growth proportional to the original table size much more quickly than
> larger tables and that could have huge consequences for queries to
> small tables which are not indexed sufficiently to handle being
> becoming bloated and large.
I'm arguing that it works well with autovacuum. Not saying there aren't
going to be certain workloads that it's suboptimal for. We're talking
about sorting by (M)XID age. As the clock continues to move forward any
table that doesn't get processed naturally moves up the queue for the
next autovac run. I think the concerns are minimal here and this would
be a good change in general.
-Jeremy
--
To know the thoughts and deeds that have marked man's progress is to
feel the great heart throbs of humanity through the centuries; and if
one does not feel in these pulsations a heavenward striving, one must
indeed be deaf to the harmonies of life.
Helen Keller, The Story Of My Life, 1902, 1903, 1905, introduction by
Ralph Barton Perry (Garden City, NY: Doubleday & Company, 1954), p90.
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
@ 2025-10-09 00:30 ` Jeremy Schneider <schneider@ardentperf.com>
1 sibling, 0 replies; 150+ messages in thread
From: Jeremy Schneider @ 2025-10-09 00:30 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Wed, 8 Oct 2025 17:27:27 -0700
Jeremy Schneider <schneider@ardentperf.com> wrote:
> On Thu, 9 Oct 2025 12:59:23 +1300
> David Rowley <dgrowleyml@gmail.com> wrote:
>
> > I believe that is methodology for processing work applies much
> > better in scenarios where there's no new work continually arriving
> > and there's no adverse effects from giving a lower priority to
> > certain portions of the work. I don't think you can apply that so
> > easily to autovacuum as there are scenarios where the work can pile
> > up faster than it can be handled. Also, smaller tables can bloat
> > in terms of growth proportional to the original table size much
> > more quickly than larger tables and that could have huge
> > consequences for queries to small tables which are not indexed
> > sufficiently to handle being becoming bloated and large.
>
> I'm arguing that it works well with autovacuum. Not saying there
> aren't going to be certain workloads that it's suboptimal for. We're
> talking about sorting by (M)XID age. As the clock continues to move
> forward any table that doesn't get processed naturally moves up the
> queue for the next autovac run. I think the concerns are minimal here
> and this would be a good change in general.
Hmm, doesn't work quite like that if the full queue needs to be
processed before the next iteration ~ but at steady state these small
tables are going to get processed at the same rate whether they were
top of bottom of the queue right?
And in non-steady-state conditions, this seems like a better order than
pg_class ordering?
-Jeremy
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
@ 2025-10-09 01:03 ` David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
1 sibling, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-09 01:03 UTC (permalink / raw)
To: Jeremy Schneider <schneider@ardentperf.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Thu, 9 Oct 2025 at 13:27, Jeremy Schneider <schneider@ardentperf.com> wrote:
> I'm arguing that it works well with autovacuum. Not saying there aren't
> going to be certain workloads that it's suboptimal for. We're talking
> about sorting by (M)XID age. As the clock continues to move forward any
> table that doesn't get processed naturally moves up the queue for the
> next autovac run. I think the concerns are minimal here and this would
> be a good change in general.
I thought if we're to have a priority queue that it would be hard to
argue against sorting by how far over the given auto-vacuum threshold
that the table is. If you assume that a table that just meets the
dead rows required to trigger autovacuum based on the
autovacuum_vacuum_scale_factor setting gets a priority of 1.0, but
another table that has n_mod_since_analyze twice over the
autovacuum_analyze_scale_factor gets priority 2.0. Effectively,
prioritise by the percentage over the given threshold the table is.
That way users could still tune things when they weren't happy with
the priority given to a table by adjusting the corresponding
reloption.
It just seems strange to me to only account for 1 of the 4 trigger
points for autovacuum when it's possible to account for all 4 without
much extra trouble.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-09 01:25 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
0 siblings, 1 reply; 150+ messages in thread
From: Jeremy Schneider @ 2025-10-09 01:25 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Thu, 9 Oct 2025 14:03:34 +1300
David Rowley <dgrowleyml@gmail.com> wrote:
> I thought if we're to have a priority queue that it would be hard to
> argue against sorting by how far over the given auto-vacuum threshold
> that the table is. If you assume that a table that just meets the
> dead rows required to trigger autovacuum based on the
> autovacuum_vacuum_scale_factor setting gets a priority of 1.0, but
> another table that has n_mod_since_analyze twice over the
> autovacuum_analyze_scale_factor gets priority 2.0. Effectively,
> prioritise by the percentage over the given threshold the table is.
> That way users could still tune things when they weren't happy with
> the priority given to a table by adjusting the corresponding
> reloption.
If users are tuning this thing then I feel like we've already lost the
battle :)
On a healthy system, autovac runs continually and hits tables at
regular intervals based on their steady state change rates. We have
existing knobs (for better or worse) that people can use to tell PG to
hit certain tables more frequently, to get rid of sleeps/delays, etc.
With our fleet of PG databases here, my current approach is geared
toward setting log_autovacuum_min_duration to some conservative value
fleet-wide, then monitoring based on the logs for any cases where it
runs longer than a defined threshold. I'm able to catch problems sooner
this way, versus monitoring on xid age alone.
Whenever there are problems with autovacuum, the actual issue is never
going to be resolved by what order autovacuum processes tables. I don't
think we should encourage any tunables here... to me it seems like
putting focus entirely in the wrong place.
-Jeremy
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
@ 2025-10-09 01:47 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Jeremy Schneider @ 2025-10-09 01:47 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Wed, 8 Oct 2025 18:25:20 -0700
Jeremy Schneider <schneider@ardentperf.com> wrote:
> On Thu, 9 Oct 2025 14:03:34 +1300
> David Rowley <dgrowleyml@gmail.com> wrote:
>
> > I thought if we're to have a priority queue that it would be hard to
> > argue against sorting by how far over the given auto-vacuum
> > threshold that the table is. If you assume that a table that just
> > meets the dead rows required to trigger autovacuum based on the
> > autovacuum_vacuum_scale_factor setting gets a priority of 1.0, but
> > another table that has n_mod_since_analyze twice over the
> > autovacuum_analyze_scale_factor gets priority 2.0. Effectively,
> > prioritise by the percentage over the given threshold the table is.
> > That way users could still tune things when they weren't happy with
> > the priority given to a table by adjusting the corresponding
> > reloption.
>
> If users are tuning this thing then I feel like we've already lost the
> battle :)
I replied too quickly. Re-reading your email, I think your proposing a
different algorithm, taking tuple counts into account. No tunables. Is
there a fully fleshed out version of the proposed alternative algorithm
somewhere? (one of the older threads?) I guess this is why its so hard
to get anything committed in this area...
-J
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
@ 2025-10-09 03:13 ` David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-09 03:13 UTC (permalink / raw)
To: Jeremy Schneider <schneider@ardentperf.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Thu, 9 Oct 2025 at 14:47, Jeremy Schneider <schneider@ardentperf.com> wrote:
>
> On Wed, 8 Oct 2025 18:25:20 -0700
> Jeremy Schneider <schneider@ardentperf.com> wrote:
> > If users are tuning this thing then I feel like we've already lost the
> > battle :)
>
> I replied too quickly. Re-reading your email, I think your proposing a
> different algorithm, taking tuple counts into account. No tunables. Is
> there a fully fleshed out version of the proposed alternative algorithm
> somewhere? (one of the older threads?) I guess this is why its so hard
> to get anything committed in this area...
It's along the lines of the "1a)" from [1]. I don't think that post
does a great job of explaining it.
I think the best way to understand it is if you look at
relation_needs_vacanalyze() and see how it calculates boolean values
for boolean output params. So, instead of calculating just a boolean
value it instead calculates a float4 where < 1.0 means don't do the
operation and anything >= 1.0 means do the operation. For example,
let's say a table has 600 dead rows and the scale factor and threshold
settings mean that autovacuum will trigger at 200 (3 times more dead
tuples than the trigger point). That would result in the value of 3.0
(600 / 200). The priority for relfrozenxid portion is basically
age(relfrozenxid) / autovacuum_freeze_max_age (plus need to account
for mxid by doing the same for that and taking the maximum of each
value). For each of those component "scores", the priority for
autovacuum would be the maximum of each of those.
Effectively, it's a method of aligning the different units of measure,
transactions or tuples into a single value which is calculated based
on the very same values that we use today to trigger autovacuums.
David
[1] https://postgr.es/m/CAApHDvo8DWyt4CWhF=NPeRstz_78SteEuuNDfYO7cjp=7YTK4g@mail.gmail.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-09 16:13 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-09 16:13 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Thu, Oct 09, 2025 at 04:13:23PM +1300, David Rowley wrote:
> I think the best way to understand it is if you look at
> relation_needs_vacanalyze() and see how it calculates boolean values
> for boolean output params. So, instead of calculating just a boolean
> value it instead calculates a float4 where < 1.0 means don't do the
> operation and anything >= 1.0 means do the operation. For example,
> let's say a table has 600 dead rows and the scale factor and threshold
> settings mean that autovacuum will trigger at 200 (3 times more dead
> tuples than the trigger point). That would result in the value of 3.0
> (600 / 200). The priority for relfrozenxid portion is basically
> age(relfrozenxid) / autovacuum_freeze_max_age (plus need to account
> for mxid by doing the same for that and taking the maximum of each
> value). For each of those component "scores", the priority for
> autovacuum would be the maximum of each of those.
>
> Effectively, it's a method of aligning the different units of measure,
> transactions or tuples into a single value which is calculated based
> on the very same values that we use today to trigger autovacuums.
I like the idea of a "score" approach, but I'm worried that we'll never
come to an agreement on the formula to use. Perhaps we'd have more luck
getting consensus on a multifaceted strategy if we kept it brutally simple.
IMHO it's worth a try...
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-10 17:31 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-10 17:31 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Thu, Oct 09, 2025 at 11:13:48AM -0500, Nathan Bossart wrote:
> On Thu, Oct 09, 2025 at 04:13:23PM +1300, David Rowley wrote:
>> I think the best way to understand it is if you look at
>> relation_needs_vacanalyze() and see how it calculates boolean values
>> for boolean output params. So, instead of calculating just a boolean
>> value it instead calculates a float4 where < 1.0 means don't do the
>> operation and anything >= 1.0 means do the operation. For example,
>> let's say a table has 600 dead rows and the scale factor and threshold
>> settings mean that autovacuum will trigger at 200 (3 times more dead
>> tuples than the trigger point). That would result in the value of 3.0
>> (600 / 200). The priority for relfrozenxid portion is basically
>> age(relfrozenxid) / autovacuum_freeze_max_age (plus need to account
>> for mxid by doing the same for that and taking the maximum of each
>> value). For each of those component "scores", the priority for
>> autovacuum would be the maximum of each of those.
>>
>> Effectively, it's a method of aligning the different units of measure,
>> transactions or tuples into a single value which is calculated based
>> on the very same values that we use today to trigger autovacuums.
>
> I like the idea of a "score" approach, but I'm worried that we'll never
> come to an agreement on the formula to use. Perhaps we'd have more luck
> getting consensus on a multifaceted strategy if we kept it brutally simple.
> IMHO it's worth a try...
Here's a prototype of a "score" approach. Two notes:
* I've given special priority to anti-wraparound vacuums. I think this is
important to avoid focusing too much on bloat when wraparound is imminent.
In any case, we need a separate wraparound score in case autovacuum is
disabled.
* I didn't include the analyze threshold in the score because it doesn't
apply to TOAST tables, and therefore would artificially lower their
prioritiy. Perhaps there is another way to deal with this.
This is very much just a prototype of the basic idea. As-is, I think it'll
favor processing tables with lots of bloat unless we're in an
anti-wraparound scenario. Maybe that's okay. I'm not sure how scientific
we want to be about all of this, but I do intend to try some long-running
tests.
--
nathan
From be37a18af33aaa436a571ead5508e89a9889b679 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v2 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 92 +++++++++++++++++++++++++----
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 81 insertions(+), 12 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index fb5d3b27224..95e928924cb 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -309,6 +309,13 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double wraparound_score;
+ double vacuum_score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +357,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *wraparound_score, double *vacuum_score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1888,6 +1896,30 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ if (t1->wraparound_score >= 1.0 || t2->wraparound_score >= 1.0)
+ {
+ if (t2->wraparound_score < t1->wraparound_score)
+ return -1;
+ else if (t2->wraparound_score > t1->wraparound_score)
+ return 1;
+ else
+ return 0;
+ }
+
+ if (t2->vacuum_score < t1->vacuum_score)
+ return -1;
+ else if (t2->vacuum_score > t2->vacuum_score)
+ return 1;
+ else
+ return 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1901,7 +1933,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2013,6 +2045,8 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double wraparound_score = 0.0;
+ double vacuum_score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2053,11 +2087,20 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &wraparound_score, &vacuum_score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->wraparound_score = wraparound_score;
+ table->vacuum_score = vacuum_score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2113,6 +2156,8 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double wraparound_score = 0.0;
+ double vacuum_score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2145,11 +2190,20 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &wraparound_score, &vacuum_score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->wraparound_score = wraparound_score;
+ table->vacuum_score = vacuum_score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2273,6 +2327,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2301,9 +2357,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2534,7 +2590,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2926,6 +2982,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double wraparound_score;
+ double vacuum_score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2933,7 +2991,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &wraparound_score, &vacuum_score);
/* Release tabentry to avoid leakage */
if (tabentry)
@@ -2992,7 +3051,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *wraparound_score, double *vacuum_score)
{
bool force_vacuum;
bool av_enabled;
@@ -3092,6 +3152,10 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ *wraparound_score = (double) relfrozenxid / freeze_max_age;
+ *wraparound_score = Max(*wraparound_score,
+ (double) classForm->relminmxid / multixact_freeze_max_age);
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
{
@@ -3165,6 +3229,10 @@ relation_needs_vacanalyze(Oid relid,
*dovacuum = force_vacuum || (vactuples > vacthresh) ||
(vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
*doanalyze = (anltuples > anlthresh);
+
+ *vacuum_score = *wraparound_score;
+ *vacuum_score += (double) vactuples / vacthresh;
+ *vacuum_score += (double) instuples / vacinsthresh;
}
else
{
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 5290b91e83e..9ba84250926 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3006,6 +3006,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v2-0001-autovacuum-scheduling-improvements.patch (7.4K, ../../aOlC4aDoQcgW8ZpC@nathan/2-v2-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From be37a18af33aaa436a571ead5508e89a9889b679 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v2 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 92 +++++++++++++++++++++++++----
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 81 insertions(+), 12 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index fb5d3b27224..95e928924cb 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -309,6 +309,13 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double wraparound_score;
+ double vacuum_score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +357,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *wraparound_score, double *vacuum_score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1888,6 +1896,30 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ if (t1->wraparound_score >= 1.0 || t2->wraparound_score >= 1.0)
+ {
+ if (t2->wraparound_score < t1->wraparound_score)
+ return -1;
+ else if (t2->wraparound_score > t1->wraparound_score)
+ return 1;
+ else
+ return 0;
+ }
+
+ if (t2->vacuum_score < t1->vacuum_score)
+ return -1;
+ else if (t2->vacuum_score > t2->vacuum_score)
+ return 1;
+ else
+ return 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1901,7 +1933,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2013,6 +2045,8 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double wraparound_score = 0.0;
+ double vacuum_score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2053,11 +2087,20 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &wraparound_score, &vacuum_score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->wraparound_score = wraparound_score;
+ table->vacuum_score = vacuum_score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2113,6 +2156,8 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double wraparound_score = 0.0;
+ double vacuum_score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2145,11 +2190,20 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &wraparound_score, &vacuum_score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->wraparound_score = wraparound_score;
+ table->vacuum_score = vacuum_score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2273,6 +2327,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2301,9 +2357,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2534,7 +2590,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2926,6 +2982,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double wraparound_score;
+ double vacuum_score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2933,7 +2991,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &wraparound_score, &vacuum_score);
/* Release tabentry to avoid leakage */
if (tabentry)
@@ -2992,7 +3051,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *wraparound_score, double *vacuum_score)
{
bool force_vacuum;
bool av_enabled;
@@ -3092,6 +3152,10 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ *wraparound_score = (double) relfrozenxid / freeze_max_age;
+ *wraparound_score = Max(*wraparound_score,
+ (double) classForm->relminmxid / multixact_freeze_max_age);
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
{
@@ -3165,6 +3229,10 @@ relation_needs_vacanalyze(Oid relid,
*dovacuum = force_vacuum || (vactuples > vacthresh) ||
(vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
*doanalyze = (anltuples > anlthresh);
+
+ *vacuum_score = *wraparound_score;
+ *vacuum_score += (double) vactuples / vacthresh;
+ *vacuum_score += (double) instuples / vacinsthresh;
}
else
{
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 5290b91e83e..9ba84250926 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3006,6 +3006,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-10 18:42 ` Robert Haas <robertmhaas@gmail.com>
2025-10-10 19:44 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Robert Haas @ 2025-10-10 18:42 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Fri, Oct 10, 2025 at 1:31 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
> Here's a prototype of a "score" approach. Two notes:
>
> * I've given special priority to anti-wraparound vacuums. I think this is
> important to avoid focusing too much on bloat when wraparound is imminent.
> In any case, we need a separate wraparound score in case autovacuum is
> disabled.
>
> * I didn't include the analyze threshold in the score because it doesn't
> apply to TOAST tables, and therefore would artificially lower their
> prioritiy. Perhaps there is another way to deal with this.
>
> This is very much just a prototype of the basic idea. As-is, I think it'll
> favor processing tables with lots of bloat unless we're in an
> anti-wraparound scenario. Maybe that's okay. I'm not sure how scientific
> we want to be about all of this, but I do intend to try some long-running
> tests.
I think this is a reasonable starting point, although I'm surprised
that you chose to combine the sub-scores using + rather than Max.
I think it will take a lot of experimentation to figure out whether
this particular algorithm (or any other) works well in practice. My
intuition (for whatever that is worth to you, which may not be much)
is that what will anger users is cases when we ignore a horrible
problem to deal with a routine problem. Figuring out how to design the
scoring system to avoid such outcomes is the hard part of this
problem, IMHO. For this particular algorithm, the main hazards that
spring to mind for me are:
- The wraparound score can't be more than about 10, but the bloat
score could be arbitrarily large, especially for tables with few
tuples, so there may be lots of cases in which the wraparound score
has no impact on the behavior.
- The patch attempts to guard against this by disregarding the
non-wraparound portion of the score once the wraparound portion
reaches 1.0, but that results in an abrupt behavior shift at that
point. Suddenly we go from mostly ignoring the wraparound score to
entirely ignoring the bloat score. This might result in the system
abruptly ignoring tables that are bloating extremely rapidly in favor
of trying to catch up in a wraparound situation that is not yet
terribly urgent.
When I've thought about this problem -- and I can't claim to have
thought about it very hard -- it's seemed to me that we need to (1)
somehow normalize everything to somewhat similar units and (2) make
sure that severe wraparound danger always wins over every other
consideration, but mild wraparound danger can lose to severe bloat.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-10-10 19:44 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 20:24 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-10 19:44 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
Thanks for taking a look.
On Fri, Oct 10, 2025 at 02:42:57PM -0400, Robert Haas wrote:
> I think this is a reasonable starting point, although I'm surprised
> that you chose to combine the sub-scores using + rather than Max.
My thinking was that we should consider as many factors as we can in the
score, not just the worst one. If a table has medium bloat and medium
wraparound risk, should it always be lower in priority to something with
large bloat and small wraparound risk? It seems worth exploring. I am
curious why you first thought of Max.
> When I've thought about this problem -- and I can't claim to have
> thought about it very hard -- it's seemed to me that we need to (1)
> somehow normalize everything to somewhat similar units and (2) make
> sure that severe wraparound danger always wins over every other
> consideration, but mild wraparound danger can lose to severe bloat.
Agreed. I need to think about this some more. While I'm optimistic that
we could come up with some sort of normalization framework, I deperately
want to avoid super complicated formulas and GUCs, as those seem like
sure-fire ways of ensuring nothing ever gets committed.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-10 19:44 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-10 20:24 ` Robert Haas <robertmhaas@gmail.com>
2025-10-10 21:59 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
0 siblings, 1 reply; 150+ messages in thread
From: Robert Haas @ 2025-10-10 20:24 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Fri, Oct 10, 2025 at 3:44 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
> On Fri, Oct 10, 2025 at 02:42:57PM -0400, Robert Haas wrote:
> > I think this is a reasonable starting point, although I'm surprised
> > that you chose to combine the sub-scores using + rather than Max.
>
> My thinking was that we should consider as many factors as we can in the
> score, not just the worst one. If a table has medium bloat and medium
> wraparound risk, should it always be lower in priority to something with
> large bloat and small wraparound risk? It seems worth exploring. I am
> curious why you first thought of Max.
The right answer depends a good bit on how exactly you do the scoring,
but it seems to me that it would be easy to overweight secondary
problems. Consider a table with an XID age of 900m and an MXID age of
900m and another table with an XID age of 1.8b. I think it is VERY
clear that the second one is MUCH worse; but just adding things up
will make them seem equal.
> Agreed. I need to think about this some more. While I'm optimistic that
> we could come up with some sort of normalization framework, I deperately
> want to avoid super complicated formulas and GUCs, as those seem like
> sure-fire ways of ensuring nothing ever gets committed.
IMHO, the trick here is to come up with something that's neither too
simple nor too complicated. If it's too simple, we'll easily come up
with cases where it sucks, and possibly where it's worse than what we
do now (an impressive achievement, to be sure). If it's too
complicated, it will be full of arbitrary things that will provoke
dissent and probably not work out well in practice. I don't think we
need something dramatically awesome to make a change to the status
quo, but if it's extremely easy to think up simple scenarios in which
a given idea will fail spectacularly, I'd be inclined to suspect that
there will be a lot of real-world spectacular failures.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-10 19:44 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 20:24 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-10-10 21:59 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-13 12:32 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Jeremy Schneider @ 2025-10-10 21:59 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Fri, 10 Oct 2025 16:24:51 -0400
Robert Haas <robertmhaas@gmail.com> wrote:
> I don't think we
> need something dramatically awesome to make a change to the status
> quo, but if it's extremely easy to think up simple scenarios in which
> a given idea will fail spectacularly, I'd be inclined to suspect that
> there will be a lot of real-world spectacular failures.
What does a real-world spectacular failure look like?
"If those 3 autovac workers had processed tables in a different order
everything would have been peachy"
But if autovac is going to get jammed up long enough to wraparound the
system, does it matter whether or not it did a one-time processing of a
bunch of small tables before it got jammed?
One particular table always scoring high shouldn't block autovac from
other tables, because it doesn't start a new iteration until it goes all
the way through the list from its current iteration right? And one
iteration of autovac needs to process everything in the list... so it
should take the same overall time regardless of order?
The spectacular failures I've seen with autovac usually come down to
things like too much sleeping (cost_delay) or too few workers, where
better ordering would be nice but probably wouldn't fix any real
problems leading to the spectacular failures
From Robert's 2024 pgConf.dev talk:
1. slow - forward progress not fast enough
2. stuck - no forward progress
3. spinning - not accomplishing anything
4. skipped - thinks not needed
5. starvation - cant keep up
I don't think any of these are really addressed by simply changing
table order.
From Robert's 2022 email to hackers:
> A few people have proposed scoring systems, which I think is closer
> to the right idea, because our basic goal is to start vacuuming any
> given table soon enough that we finish vacuuming it before some
> catastrophe strikes.
...
> If table A will cause wraparound in 2 hours and take 2 hours to
> vacuum, and table B will cause wraparound in 1 hour and take 10
> minutes to vacuum, table A is more urgent even though the catastrophe
> is further out.
Robert it sounds to me like the main use case you're focused on here
is where basically wraparound is imminent - we are already screwed - and
our very last hope was that a last-ditch autovac can finish just in time
Failsafe and dynamic cost updates were huge advancements. Do we allow
dynamic adjustment to worker count yet?
I hope y'all just pick something and commit it without getting too lost
in the details. I honestly think in the list of improvements around
autovac, this is the lowest priority on my list of hopes and dreams as a
user for wraparound prevention :) because if this ever matters to me for
avoiding wraparound, I was screwed long before we got to this point and
this is not going to fix my underlying problems.
-Jeremy
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-10 19:44 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 20:24 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-10 21:59 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
@ 2025-10-13 12:32 ` Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 150+ messages in thread
From: Robert Haas @ 2025-10-13 12:32 UTC (permalink / raw)
To: Jeremy Schneider <schneider@ardentperf.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Fri, Oct 10, 2025 at 6:00 PM Jeremy Schneider
<schneider@ardentperf.com> wrote:
> The spectacular failures I've seen with autovac usually come down to
> things like too much sleeping (cost_delay) or too few workers, where
> better ordering would be nice but probably wouldn't fix any real
> problems leading to the spectacular failures
Since I have said the same thing myself, I can hardly disagree.
However, there are probably a few exceptions. For instance, if
autovacuum on a certain table is failing repeatedly or accomplishing
nothing without removing the apparent need to autovacuum, and happens
to be the first one in pg_class, it could divert a lot of attention
from other tables.
> Robert it sounds to me like the main use case you're focused on here
> is where basically wraparound is imminent - we are already screwed - and
> our very last hope was that a last-ditch autovac can finish just in time
Yes, I would argue that this is the scenario that really matters. As
you say above, the main thing is having little enough sleeping and a
sufficient number of workers. When that's the case, we can do the work
in any order and life will mostly be fine. However, if we get into a
desperate situation by, say, having one table that can't be vacuumed,
and eventually someone fixes that, say by dropping the corrupt index
that is preventing vacuuming of that table, we might like it if
autovacuum focused on getting that table vacuumed rather than getting
lost in the sauce. Of course, if we have the pretty common situation
where autovacuum gets behind on all tables, say due to a stale
replication slot, then this is less critical, although a perfect
system would probably prioritize vacuuming the *largest* tables in
this situation, since those will take the longest to finish, and it's
when a vacuum of every table in the cluster has been *completed* that
the XID horizons can advance.
> I hope y'all just pick something and commit it without getting too lost
> in the details. I honestly think in the list of improvements around
> autovac, this is the lowest priority on my list of hopes and dreams as a
> user for wraparound prevention :) because if this ever matters to me for
> avoiding wraparound, I was screwed long before we got to this point and
> this is not going to fix my underlying problems.
I'm not sure if this was your intention, but to me this kind of reads
like "well, it's not going to matter anyway so just do whatever and
move on" and I don't agree with that. I think that if we're not going
to do high-quality engineering here, we just shouldn't change anything
at all. It's better to keep having the same bad behavior than for each
release to have new and different bad behavior. One possible positive
result of leaning into this prioritization problem is that whoever's
working in it (Nathan, in this case) might gain some useful insights
about how to tackle some of the other problems in this space. All of
this is hard enough that we haven't really had any major improvements
in this area since, I want to say, 8.3, and it's desirable to break
that logjam even if we don't all agree on which problems are most
urgent. Even if I ultimately don't agree with whatever Nathan wants to
do or proposes, I'm glad he's trying to do something, which is (in my
experience) generally much better than making no effort at all.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-10-12 06:27 ` David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-12 06:27 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Sat, 11 Oct 2025 at 07:43, Robert Haas <robertmhaas@gmail.com> wrote:
> I think this is a reasonable starting point, although I'm surprised
> that you chose to combine the sub-scores using + rather than Max.
Adding up the component scores doesn't make sense to me either. That
means you could have 0.5 for inserted tuples, 0.5 for dead tuples and,
say 0.1 for analyze threshold, which all add up to 1.1, but neither
component score is high enough for auto-vacuum to have to do anything
yet. With Max(), we'd clearly see that there's nothing to do since the
overall score isn't >= 1.0.
> - The wraparound score can't be more than about 10, but the bloat
> score could be arbitrarily large, especially for tables with few
> tuples, so there may be lots of cases in which the wraparound score
> has no impact on the behavior.
That's a good point. I think we definitely do want to make it so
tables in near danger of causing the database to stop accepting
transactions are dealt with ASAP.
Maybe the score calculation could change when the relevant age() goes
above vacuum_failsafe_age / vacuum_multixact_failsafe_age and start
scaling it very aggressively beyond that. There's plenty to debate,
but at a first cut, maybe something like the following (coded in SQL
for ease of result viewing):
select xidage as "age(relfrozenxid)",case xidage::float8 <
current_setting('vacuum_failsafe_age')::float8 when true then xidage /
current_setting('autovacuum_freeze_max_age')::float8 else power(xidage
/ current_setting('autovacuum_freeze_max_age')::float8,xidage::float8
/ 100_000_000) end xid_age_score from
generate_series(0,2_000_000_000,100_000_000) xidage;
which gives 1e+20 for age of 2 billion. It would take quite an
unreasonable amount of bloat to score higher than that.
I guess someone might argue that we should start taking it more
seriously before the table's relfrozenxid age gets to
vacuum_failsafe_age. Maybe that's true. I just don't know what. In any
case, if a table's age gets that old, then something's probably not
configured very well and needs attention. I did think maybe we could
keep the addressing of auto-vacuum being configured to run too slowly
as a separate thread.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-21 14:38 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-21 14:38 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Sun, Oct 12, 2025 at 07:27:10PM +1300, David Rowley wrote:
> On Sat, 11 Oct 2025 at 07:43, Robert Haas <robertmhaas@gmail.com> wrote:
>> I think this is a reasonable starting point, although I'm surprised
>> that you chose to combine the sub-scores using + rather than Max.
>
> Adding up the component scores doesn't make sense to me either. That
> means you could have 0.5 for inserted tuples, 0.5 for dead tuples and,
> say 0.1 for analyze threshold, which all add up to 1.1, but neither
> component score is high enough for auto-vacuum to have to do anything
> yet. With Max(), we'd clearly see that there's nothing to do since the
> overall score isn't >= 1.0.
In v3, I switched to Max().
> Maybe the score calculation could change when the relevant age() goes
> above vacuum_failsafe_age / vacuum_multixact_failsafe_age and start
> scaling it very aggressively beyond that. There's plenty to debate,
> but at a first cut, maybe something like the following (coded in SQL
> for ease of result viewing):
>
> select xidage as "age(relfrozenxid)",case xidage::float8 <
> current_setting('vacuum_failsafe_age')::float8 when true then xidage /
> current_setting('autovacuum_freeze_max_age')::float8 else power(xidage
> / current_setting('autovacuum_freeze_max_age')::float8,xidage::float8
> / 100_000_000) end xid_age_score from
> generate_series(0,2_000_000_000,100_000_000) xidage;
>
> which gives 1e+20 for age of 2 billion. It would take quite an
> unreasonable amount of bloat to score higher than that.
>
> I guess someone might argue that we should start taking it more
> seriously before the table's relfrozenxid age gets to
> vacuum_failsafe_age. Maybe that's true. I just don't know what. In any
> case, if a table's age gets that old, then something's probably not
> configured very well and needs attention. I did think maybe we could
> keep the addressing of auto-vacuum being configured to run too slowly
> as a separate thread.
I did something similar to this in v3, although I used the *_freeze_max_age
parameters as the point to start scaling aggressively, and I simply raised
the score to the power of 10.
I've yet to do any real testing with this stuff.
--
nathan
From a54fce8b43134381e504cf3f7fd4a5d4fff8415e Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v3 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 102 ++++++++++++++++++++++++----
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 88 insertions(+), 15 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..075d4e87a86 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,7 +2979,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
@@ -3000,7 +3039,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,7 +3069,10 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
+ double xid_score;
+ double mxid_score;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
@@ -3081,17 +3124,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3100,6 +3143,23 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by its
+ * respective *_freeze_max_age parameter.
+ */
+ xid_score = (double) relfrozenxid / freeze_max_age;
+ mxid_score = (double) relminmxid / multixact_freeze_max_age;
+ *score = Max(xid_score, mxid_score);
+
+ /*
+ * To ensure tables are given increased priority once they surpass their
+ * *_freeze_max_age parameters, we raise it to the power of 10. This
+ * ensures that the age becomes the main priority the closer we are to
+ * wraparound.
+ */
+ if (*score > 1.0)
+ *score = pow(*score, 10.0);
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
{
@@ -3173,6 +3233,18 @@ relation_needs_vacanalyze(Oid relid,
*dovacuum = force_vacuum || (vactuples > vacthresh) ||
(vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
*doanalyze = (anltuples > anlthresh);
+
+ /*
+ * Update the score, being careful to avoid division by zero and to
+ * skip cases where auto-analyze does not apply, i.e., pg_statistic
+ * and TOAST tables.
+ */
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ if (relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ if (vac_ins_base_thresh >= 0)
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
}
else
{
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 377a7946585..3389cf5ebae 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3007,6 +3007,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v3-0001-autovacuum-scheduling-improvements.patch (8.7K, ../../aPea8dG8_lFwTsyo@nathan/2-v3-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From a54fce8b43134381e504cf3f7fd4a5d4fff8415e Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v3 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 102 ++++++++++++++++++++++++----
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 88 insertions(+), 15 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..075d4e87a86 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,7 +2979,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
@@ -3000,7 +3039,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,7 +3069,10 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
+ double xid_score;
+ double mxid_score;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
@@ -3081,17 +3124,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3100,6 +3143,23 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by its
+ * respective *_freeze_max_age parameter.
+ */
+ xid_score = (double) relfrozenxid / freeze_max_age;
+ mxid_score = (double) relminmxid / multixact_freeze_max_age;
+ *score = Max(xid_score, mxid_score);
+
+ /*
+ * To ensure tables are given increased priority once they surpass their
+ * *_freeze_max_age parameters, we raise it to the power of 10. This
+ * ensures that the age becomes the main priority the closer we are to
+ * wraparound.
+ */
+ if (*score > 1.0)
+ *score = pow(*score, 10.0);
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
{
@@ -3173,6 +3233,18 @@ relation_needs_vacanalyze(Oid relid,
*dovacuum = force_vacuum || (vactuples > vacthresh) ||
(vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
*doanalyze = (anltuples > anlthresh);
+
+ /*
+ * Update the score, being careful to avoid division by zero and to
+ * skip cases where auto-analyze does not apply, i.e., pg_statistic
+ * and TOAST tables.
+ */
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ if (relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ if (vac_ins_base_thresh >= 0)
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
}
else
{
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 377a7946585..3389cf5ebae 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3007,6 +3007,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-21 20:07 ` David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-21 20:07 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Wed, 22 Oct 2025 at 03:38, Nathan Bossart <nathandbossart@gmail.com> wrote:
> I did something similar to this in v3, although I used the *_freeze_max_age
> parameters as the point to start scaling aggressively, and I simply raised
> the score to the power of 10.
>
> I've yet to do any real testing with this stuff.
I've not tested it or compiled it, but the patch looks good. I did
think that the freeze vacuum isn't that big a deal if it's just over
the *freeze_max_age and thought it should become aggressive very
quickly at the failsafe age, but that leaves a much smaller window of
time to do the freezing if autovacuum has been busy with other higher
priority tables. Your scaling is much more gentle and comes out (with
standard settings) with a score of 1 billion for a table at the
failsafe age, and about 1 million at half the failsafe age. That seems
reasonable as it's hard to imagine a table having a 1 billion bloat
score.
However, just thinking of non-standard setting... I do wonder if it'll
be aggressive enough if someone did something like raise the
*freeze_max_age to 1 billion (it's certainly common that people raise
this). With a 1.6 billion vacuum_failsafe_age, a table at
freeze_max_age only scores in at 110. I guess there's no reason we
couldn't keep your calc and then scale the score further once over
vacuum_failsafe_age to ensure those are the highest priority. There is
a danger that if a table scores too low when age(relfrozenxid) >
vacuum_failsafe_age that autovacuum dawdles along handling bloated
tables while oblivious to the nearing armageddon.
Is it worth writing a comment explaining the philosophy behind the
scoring system to make it easier for people to understand that it aims
to standardise the priority of vacuums and unify the various trigger
thresholds into a single number to determine which tables are most
important to vacuum and/or analyze first?
Thanks for working on this.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-22 18:40 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-22 18:40 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Wed, Oct 22, 2025 at 09:07:33AM +1300, David Rowley wrote:
> However, just thinking of non-standard setting... I do wonder if it'll
> be aggressive enough if someone did something like raise the
> *freeze_max_age to 1 billion (it's certainly common that people raise
> this). With a 1.6 billion vacuum_failsafe_age, a table at
> freeze_max_age only scores in at 110. I guess there's no reason we
> couldn't keep your calc and then scale the score further once over
> vacuum_failsafe_age to ensure those are the highest priority. There is
> a danger that if a table scores too low when age(relfrozenxid) >
> vacuum_failsafe_age that autovacuum dawdles along handling bloated
> tables while oblivious to the nearing armageddon.
That's a good point. I wonder if we should try to make the wraparound
score independent of the *_freeze_max_age parameters (once the table age
surpasses said parameters). Else, different settings will greatly impact
how aggressively tables are prioritized the closer they are to wraparound.
Even if autovacuum_freeze_max_age is set to 200M, it's not critically
important for autovacuum to pick up tables right away as soon as their age
reaches 200M. But if the parameter is set to 2B, we _do_ want autovacuum
to prioritize tables right away once their age reaches 2B.
> Is it worth writing a comment explaining the philosophy behind the
> scoring system to make it easier for people to understand that it aims
> to standardise the priority of vacuums and unify the various trigger
> thresholds into a single number to determine which tables are most
> important to vacuum and/or analyze first?
Yes, I think so.
> Thanks for working on this.
I appreciate the discussion.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-22 18:58 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-22 18:58 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Wed, Oct 22, 2025 at 01:40:11PM -0500, Nathan Bossart wrote:
> On Wed, Oct 22, 2025 at 09:07:33AM +1300, David Rowley wrote:
>> However, just thinking of non-standard setting... I do wonder if it'll
>> be aggressive enough if someone did something like raise the
>> *freeze_max_age to 1 billion (it's certainly common that people raise
>> this). With a 1.6 billion vacuum_failsafe_age, a table at
>> freeze_max_age only scores in at 110. I guess there's no reason we
>> couldn't keep your calc and then scale the score further once over
>> vacuum_failsafe_age to ensure those are the highest priority. There is
>> a danger that if a table scores too low when age(relfrozenxid) >
>> vacuum_failsafe_age that autovacuum dawdles along handling bloated
>> tables while oblivious to the nearing armageddon.
>
> That's a good point. I wonder if we should try to make the wraparound
> score independent of the *_freeze_max_age parameters (once the table age
> surpasses said parameters). Else, different settings will greatly impact
> how aggressively tables are prioritized the closer they are to wraparound.
> Even if autovacuum_freeze_max_age is set to 200M, it's not critically
> important for autovacuum to pick up tables right away as soon as their age
> reaches 200M. But if the parameter is set to 2B, we _do_ want autovacuum
> to prioritize tables right away once their age reaches 2B.
I'm imagining something a bit like the following:
select xidage "age(relfrozenxid)",
power(1.001, xidage::float8 / (select min_val
from pg_settings where name = 'autovacuum_freeze_max_age')::float8)
xid_age_score from generate_series(0,2_000_000_000,100_000_000) xidage;
age(relfrozenxid) | xid_age_score
-------------------+--------------------
0 | 1
100000000 | 2.7169239322355936
200000000 | 7.38167565355452
300000000 | 20.055451243143093
400000000 | 54.48913545427955
500000000 | 148.0428361625591
600000000 | 402.22112456608977
700000000 | 1092.804199384323
800000000 | 2969.065882554825
900000000 | 8066.726152697397
1000000000 | 21916.681339054314
1100000000 | 59545.956045257895
1200000000 | 161781.8330472099
1300000000 | 439548.9340069078
1400000000 | 1194221.0181920114
1500000000 | 3244607.664704634
1600000000 | 8815352.21495106
1700000000 | 23950641.403886583
1800000000 | 65072070.82261215
1900000000 | 176795866.53808445
2000000000 | 480340920.9176516
(21 rows)
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-22 19:34 ` David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-24 21:13 ` Re: another autovacuum scheduling thread Peter Geoghegan <pg@bowt.ie>
0 siblings, 2 replies; 150+ messages in thread
From: David Rowley @ 2025-10-22 19:34 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Thu, 23 Oct 2025 at 07:58, Nathan Bossart <nathandbossart@gmail.com> wrote:
> > That's a good point. I wonder if we should try to make the wraparound
> > score independent of the *_freeze_max_age parameters (once the table age
> > surpasses said parameters). Else, different settings will greatly impact
> > how aggressively tables are prioritized the closer they are to wraparound.
> > Even if autovacuum_freeze_max_age is set to 200M, it's not critically
> > important for autovacuum to pick up tables right away as soon as their age
> > reaches 200M. But if the parameter is set to 2B, we _do_ want autovacuum
> > to prioritize tables right away once their age reaches 2B.
>
> I'm imagining something a bit like the following:
>
> select xidage "age(relfrozenxid)",
> power(1.001, xidage::float8 / (select min_val
> from pg_settings where name = 'autovacuum_freeze_max_age')::float8)
> xid_age_score from generate_series(0,2_000_000_000,100_000_000) xidage;
>
> age(relfrozenxid) | xid_age_score
> -------------------+--------------------
> 0 | 1
> 100000000 | 2.7169239322355936
> 200000000 | 7.38167565355452
> 300000000 | 20.055451243143093
This does start to put the score > 1 before the table reaches
autovacuum_freeze_max_age. I don't think that's great as the score of
1.0 was meant to represent that the table now requires some autovacuum
work.
The main reason I was trying to keep the score scaling with the
percentage over the given threshold that the table is was that I had
imagined we could use the score number to start reducing the sleep
time between autovacuum_vacuum_cost_limit when the highest scoring
table persists in being high for too long. I was considering this to
fix the misconfigured autovacuum problem that so many people have. If
we scaled it the way similar to the query above, the score would look
high even before it reaches the limit. This is the reason I was
scaling the score linear with the autovacuum_freeze_max_age with the
version I sent and only scaling exponentially after the failsafe age.
I wanted to talk about the "reducing the cost delay" feature
separately so as not to load up this thread and widen the scope for
varying opinions, but in its most trivial form, the
vacuum_cost_limit() code could be adjusted to only sleep for
autovacuum_vacuum_cost_delay / <the table's score>.
I think the one I proposed in [1] does this quite well. The table
remains eligible to be autovacuumed with any score >= 1.0, and there's
still a huge window of time to freeze a table once it's over
autovacuum_freeze_max_age before there are issues and the exponential
scaling once over failsafe age should ensure that the table is top of
the list for when the failsafe code kicks in and removes the cost
limit. If we had the varying sleep time as I mentioned above, the
failsafe code could even be removed as the
"autovacuum_vacuum_cost_delay / <tables score>" calculation would
effectively zero the sleep time with any table > failsafe age.
David
[1] https://postgr.es/m/CAApHDvqrd=SHVUytdRj55OWnLH98Rvtzqam5zq2f4XKRZa7t9Q@mail.gmail.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-22 19:43 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-22 19:43 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Thu, Oct 23, 2025 at 08:34:49AM +1300, David Rowley wrote:
> On Thu, 23 Oct 2025 at 07:58, Nathan Bossart <nathandbossart@gmail.com> wrote:
>> I'm imagining something a bit like the following:
>>
>> select xidage "age(relfrozenxid)",
>> power(1.001, xidage::float8 / (select min_val
>> from pg_settings where name = 'autovacuum_freeze_max_age')::float8)
>> xid_age_score from generate_series(0,2_000_000_000,100_000_000) xidage;
>>
>> age(relfrozenxid) | xid_age_score
>> -------------------+--------------------
>> 0 | 1
>> 100000000 | 2.7169239322355936
>> 200000000 | 7.38167565355452
>> 300000000 | 20.055451243143093
>
> This does start to put the score > 1 before the table reaches
> autovacuum_freeze_max_age. I don't think that's great as the score of
> 1.0 was meant to represent that the table now requires some autovacuum
> work.
My thinking was that this formula would only be used once the table reaches
autovacuum_freeze_max_age. If the age is less than that, we'd do something
else, such as dividing the age by the *_max_age setting.
> The main reason I was trying to keep the score scaling with the
> percentage over the given threshold that the table is was that I had
> imagined we could use the score number to start reducing the sleep
> time between autovacuum_vacuum_cost_limit when the highest scoring
> table persists in being high for too long. I was considering this to
> fix the misconfigured autovacuum problem that so many people have. If
> we scaled it the way similar to the query above, the score would look
> high even before it reaches the limit. This is the reason I was
> scaling the score linear with the autovacuum_freeze_max_age with the
> version I sent and only scaling exponentially after the failsafe age.
> I wanted to talk about the "reducing the cost delay" feature
> separately so as not to load up this thread and widen the scope for
> varying opinions, but in its most trivial form, the
> vacuum_cost_limit() code could be adjusted to only sleep for
> autovacuum_vacuum_cost_delay / <the table's score>.
I see.
> I think the one I proposed in [1] does this quite well. The table
> remains eligible to be autovacuumed with any score >= 1.0, and there's
> still a huge window of time to freeze a table once it's over
> autovacuum_freeze_max_age before there are issues and the exponential
> scaling once over failsafe age should ensure that the table is top of
> the list for when the failsafe code kicks in and removes the cost
> limit.
Yeah. I'll update the patch with that formula.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-23 18:22 ` Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-10-23 18:22 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> > I think the one I proposed in [1] does this quite well. The table
> > remains eligible to be autovacuumed with any score >= 1.0, and there's
> > still a huge window of time to freeze a table once it's over
> > autovacuum_freeze_max_age before there are issues and the exponential
> > scaling once over failsafe age should ensure that the table is top of
> > the list for when the failsafe code kicks in and removes the cost
> > limit.
>
> Yeah. I'll update the patch with that formula.
I was looking at v3, and I understand the formula will be updated in the
next version. However, do you think we should benchmark the approach
of using an intermediary list to store the eligible tables and sorting
that list,
which may cause larger performance overhead for databases with hundreds
of tables that may all be eligible for autovacuum. I do think such cases
out there are common, particularly in multi-tenant type databases, where
each tenant could be one or more tables.
What do you think?
--
Sami
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-23 18:47 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-23 18:47 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Oct 23, 2025 at 01:22:24PM -0500, Sami Imseih wrote:
> I was looking at v3, and I understand the formula will be updated in the
> next version. However, do you think we should benchmark the approach
> of using an intermediary list to store the eligible tables and sorting
> that list,
> which may cause larger performance overhead for databases with hundreds
> of tables that may all be eligible for autovacuum. I do think such cases
> out there are common, particularly in multi-tenant type databases, where
> each tenant could be one or more tables.
We already have an intermediary list of table OIDs, so the additional
overhead is ultimately just the score calculation and the sort operation.
I'd be quite surprised if that added up to anything remotely worrisome,
even for thousands of eligible tables.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-23 19:32 ` Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-10-23 19:32 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> On Thu, Oct 23, 2025 at 01:22:24PM -0500, Sami Imseih wrote:
> > I was looking at v3, and I understand the formula will be updated in the
> > next version. However, do you think we should benchmark the approach
> > of using an intermediary list to store the eligible tables and sorting
> > that list,
> > which may cause larger performance overhead for databases with hundreds
> > of tables that may all be eligible for autovacuum. I do think such cases
> > out there are common, particularly in multi-tenant type databases, where
> > each tenant could be one or more tables.
>
> We already have an intermediary list of table OIDs, so the additional
> overhead is ultimately just the score calculation and the sort operation.
> I'd be quite surprised if that added up to anything remotely worrisome,
> even for thousands of eligible tables.
Yeah, you’re correct, the list already exists; sorry I missed that. My
main concern is
the additional overhead of the sort operation, especially if we have
many eligible
tables and an aggressive autovacuum_naptime. I don’t think we should make the
existing performance of many relations any worse with an additional
sort. That said,
in such cases the sort may not even be the main performance
bottleneck, since the
catalog scan itself already doesn’t scale well with many relations.
With our current
approach, we have more options to improve this, but if we add a sort,
we may not be
able to avoid a full scan.
--
Sami
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-23 20:24 ` David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-23 20:24 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, 24 Oct 2025 at 08:33, Sami Imseih <samimseih@gmail.com> wrote:
> Yeah, you’re correct, the list already exists; sorry I missed that. My
> main concern is
> the additional overhead of the sort operation, especially if we have
> many eligible
> tables and an aggressive autovacuum_naptime.
It is true that there are reasons that millions of tables could
suddenly become eligible for autovacuum work with the consumption of a
single xid, but I imagine sorting the list of tables is probably the
least of the DBAs worries for that case as sorting the
tables_to_process list is going to take a tiny fraction of the time
that doing the vacuum work will take.
If your concern is that the sort could take too large a portion of
someone's 1sec autovacuum_naptime instance, then you also need to
consider that the list isn't likely to be very long as there's very
little time for tables to become eligible in such a short naptime, and
if the tables are piling up because autovacuum is configured to run
too slowly, then lets fix that at the root cause rather than be
worried about improving one area because another area needs work. If
we think like that, we'll remain gridlocked and autovacuum will never
be improved. TBH, I think that mindset has likely contributed quite a
bit to the fact that we've made about zero improvements in this area
despite nobody thinking that nothing needs to be done.
There are also things that could be done if we were genuinely
concerned and had actual proof that this could reasonably be a
problem. sort_template.h would reduce the constant factor of the
indirect function call overhead by quite a bit. On a quick test here
with a table containing 1 million random float8 values, a Seq Scan and
in-memory Sort, EXPLAIN ANALYZE reports the sort took about 21ms:
(actual time=172.273..193.824). I really doubt anyone will be
concerned with 21ms when there's a list of 1 million tables needing to
be autovacuumed.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-23 20:48 ` Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-10-23 20:48 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> On Fri, 24 Oct 2025 at 08:33, Sami Imseih <samimseih@gmail.com> wrote:
> > Yeah, you’re correct, the list already exists; sorry I missed that. My
> > main concern is
> > the additional overhead of the sort operation, especially if we have
> > many eligible
> > tables and an aggressive autovacuum_naptime.
>
> It is true that there are reasons that millions of tables could
> suddenly become eligible for autovacuum work with the consumption of a
> single xid, but I imagine sorting the list of tables is probably the
> least of the DBAs worries for that case as sorting the
> tables_to_process list is going to take a tiny fraction of the time
> that doing the vacuum work will take.
Yes, in my last reply, I did indicate that the sort will likely not be
the operation that will tip the performance over, but the
catalog scan itself that I have seen not scale well as the number of
relations grow ( in cases of thousands or hundreds of thousands of tables).
If we are to prioritize vacuuming by M(XID), then it will be hard to avoid the
catalog scan anymore in a future improvement.
>TBH, I think that mindset has likely contributed quite a
> bit to the fact that we've made about zero improvements in this area
> despite nobody thinking that nothing needs to be done.
I am not against this idea, just thinking out loud about the high relation
cases I have seen in the past.
--
Sami
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-23 22:39 ` David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-23 22:39 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, 24 Oct 2025 at 09:48, Sami Imseih <samimseih@gmail.com> wrote:
> Yes, in my last reply, I did indicate that the sort will likely not be
> the operation that will tip the performance over, but the
> catalog scan itself that I have seen not scale well as the number of
> relations grow ( in cases of thousands or hundreds of thousands of tables).
> If we are to prioritize vacuuming by M(XID), then it will be hard to avoid the
> catalog scan anymore in a future improvement.
I grant you that I could see that could be a problem for a
sufficiently large number of tables and small enough
autovacuum_naptime, but I don't see how anything being proposed here
moves the goalposts on the requirements to scan pg_class. We at least
need to get the relopts from somewhere, plus reltuples, relpages,
relallfrozen. We can't magic those values out of thin air. So, since
nothing is changing in regards to the scan of pg_class or which
columns we need to look at in that table, I don't know why we'd
consider it a topic to discuss on this thread. If this thread becomes
a dumping ground for unrelated problems, then nothing will be done to
fix the problem at hand.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-24 15:08 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-24 15:08 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Here is an updated patch based on the latest discussion.
--
nathan
From 232bc8a73711c743e74d964abfb202563e176aa5 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v4 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 142 +++++++++++++++++++++++++---
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 128 insertions(+), 15 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..e8cd414c445 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,7 +2979,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
@@ -2990,6 +3029,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3000,7 +3065,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,7 +3095,14 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
@@ -3081,17 +3154,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3100,6 +3173,33 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by its
+ * respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? relfrozenxid - recentXid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? relminmxid - recentMulti : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
{
@@ -3173,6 +3273,18 @@ relation_needs_vacanalyze(Oid relid,
*dovacuum = force_vacuum || (vactuples > vacthresh) ||
(vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
*doanalyze = (anltuples > anlthresh);
+
+ /*
+ * Update the score, being careful to avoid division by zero and to
+ * skip cases where auto-analyze does not apply, i.e., pg_statistic
+ * and TOAST tables.
+ */
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ if (relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ if (vac_ins_base_thresh >= 0)
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
}
else
{
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 43fe3bcd593..3e0739c7808 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3008,6 +3008,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v4-0001-autovacuum-scheduling-improvements.patch (11.3K, ../../aPuWev3D9M4iGCUt@nathan/2-v4-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From 232bc8a73711c743e74d964abfb202563e176aa5 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v4 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 142 +++++++++++++++++++++++++---
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 128 insertions(+), 15 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..e8cd414c445 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,7 +2979,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
@@ -2990,6 +3029,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3000,7 +3065,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,7 +3095,14 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
@@ -3081,17 +3154,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3100,6 +3173,33 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by its
+ * respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? relfrozenxid - recentXid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? relminmxid - recentMulti : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
{
@@ -3173,6 +3273,18 @@ relation_needs_vacanalyze(Oid relid,
*dovacuum = force_vacuum || (vactuples > vacthresh) ||
(vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
*doanalyze = (anltuples > anlthresh);
+
+ /*
+ * Update the score, being careful to avoid division by zero and to
+ * skip cases where auto-analyze does not apply, i.e., pg_statistic
+ * and TOAST tables.
+ */
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ if (relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ if (vac_ins_base_thresh >= 0)
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
}
else
{
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 43fe3bcd593..3e0739c7808 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3008,6 +3008,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-26 01:25 ` David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-26 01:25 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sat, 25 Oct 2025 at 04:08, Nathan Bossart <nathandbossart@gmail.com> wrote:
> Here is an updated patch based on the latest discussion.
Thanks. I've just had a look at it. A few comments and questions.
1) The subtraction here looks back to front:
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? relfrozenxid - recentXid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? relminmxid - recentMulti : 0;
2) Would it be better to move all the code that sets the xid_score and
mxid_score to under an "if (force_vacuum)"? Those two variables could
be declared in there too.
3) Could the following be refactored a bit so we only check the "relid
!= StatisticRelationId" condition once?
+ if (relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
Something like:
/* ANALYZE refuses to work with pg_statistic and we don't analyze
toast tables */
if (anltuples > anlthresh && relid != StatisticRelationId &&
classForm->relkind != RELKIND_TOASTVALUE)
{
*doanalyze = true;
// calc analyze score and Max with *score
}
else
*doanalyze = false;
then delete:
/* ANALYZE refuses to work with pg_statistic */
if (relid == StatisticRelationId)
*doanalyze = false;
4) Should these be TransactionIds?
+ uint32 xid_age;
+ uint32 mxid_age;
5) Instead of:
+ double score = 0.0;
Is it better to zero the score inside relation_needs_vacanalyze() so
it works the same as the other output parameters?
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-27 16:06 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Nathan Bossart @ 2025-10-27 16:06 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sun, Oct 26, 2025 at 02:25:48PM +1300, David Rowley wrote:
> Thanks. I've just had a look at it. A few comments and questions.
Thanks.
> 1) The subtraction here looks back to front:
>
> + xid_age = TransactionIdIsNormal(relfrozenxid) ? relfrozenxid - recentXid : 0;
> + mxid_age = MultiXactIdIsValid(relminmxid) ? relminmxid - recentMulti : 0;
D'oh.
> 2) Would it be better to move all the code that sets the xid_score and
> mxid_score to under an "if (force_vacuum)"? Those two variables could
> be declared in there too.
Seems reasonable.
> 3) Could the following be refactored a bit so we only check the "relid
> != StatisticRelationId" condition once?
Yes. We can update the vacuum part to follow the same pattern, too.
> 4) Should these be TransactionIds?
>
> + uint32 xid_age;
> + uint32 mxid_age;
Probably.
> 5) Instead of:
>
> + double score = 0.0;
>
> Is it better to zero the score inside relation_needs_vacanalyze() so
> it works the same as the other output parameters?
My only concern about this is that some compilers might complain about
potentially-uninitialized uses. But we can still zero it in the function
regardless.
--
nathan
From 8fc0829a2be30ea29d263d45732c6ef1bd2c6ef0 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v5 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 171 ++++++++++++++++++++++++----
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 153 insertions(+), 19 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..97ace126821 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,7 +2979,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
@@ -2990,6 +3029,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3000,7 +3065,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,11 +3095,15 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* ensure score is initialized to zero */
+ *score = 0.0;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3081,17 +3151,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3100,6 +3170,44 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
{
@@ -3169,10 +3277,35 @@ relation_needs_vacanalyze(Oid relid,
NameStr(classForm->relname),
vactuples, vacthresh, anltuples, anlthresh);
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (force_vacuum ||
+ vactuples > vacthresh ||
+ (vac_ins_base_thresh >= 0 && instuples > vacinsthresh))
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ if (vac_ins_base_thresh >= 0)
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
+ }
+ else
+ *dovacuum = false;
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ *doanalyze = true;
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ }
+ else
+ *doanalyze = false;
}
else
{
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bb4e1b37005..6606e6808d3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3007,6 +3007,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v5-0001-autovacuum-scheduling-improvements.patch (12.0K, ../../aP-YgrcPi0EhgR9x@nathan/2-v5-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From 8fc0829a2be30ea29d263d45732c6ef1bd2c6ef0 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v5 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 171 ++++++++++++++++++++++++----
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 153 insertions(+), 19 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..97ace126821 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,7 +2979,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
@@ -2990,6 +3029,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3000,7 +3065,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,11 +3095,15 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* ensure score is initialized to zero */
+ *score = 0.0;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3081,17 +3151,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3100,6 +3170,44 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
{
@@ -3169,10 +3277,35 @@ relation_needs_vacanalyze(Oid relid,
NameStr(classForm->relname),
vactuples, vacthresh, anltuples, anlthresh);
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (force_vacuum ||
+ vactuples > vacthresh ||
+ (vac_ins_base_thresh >= 0 && instuples > vacinsthresh))
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ if (vac_ins_base_thresh >= 0)
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
+ }
+ else
+ *dovacuum = false;
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ *doanalyze = true;
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ }
+ else
+ *doanalyze = false;
}
else
{
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bb4e1b37005..6606e6808d3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3007,6 +3007,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-27 17:47 ` Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-10-27 17:47 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
I spent some time looking at this, and I am not sure how much this
will move the goalpost, since most of the time the bottleneck for
autovacuum is the limited number of workers and large tables that
take a long time to process.
That said, this is a good change for the simple reason that it is
better to have a well-defined prioritization strategy for autovacuum
than something that is somewhat random, as mentioned earlier.
Just a couple of comments on v5:
1/ Should we add documentation explaining this prioritization behavior in [0]?
I wrote a sql that returns the tables and scores, which I found was
useful when I was testing this out, so having the actually rules spelled out
in docs will actually be super useful.
If we don't want to go that much in depth, at minimum the docs should say:
"Autovacuum prioritizes tables based on how far they exceed their thresholds
or if they are approaching wraparound limits." so a DBA can understand
this behavior.
2/
* The score is calculated as the maximum of the ratios of each of the table's
* relevant values to its threshold. For example, if the number of inserted
* tuples is 100, and the insert threshold for the table is 80, the insert
* score is 1.25.
Should we consider clamping down on the score when
reltuples = -1, otherwise the scores for such tables ( new tables
with a large amount of ingested data ) will be over-inflated? Perhaps,
if reltuples = -1 ( # of reltuples not known ), then give a score of .5,
so we are not over-prioritizing but not pushing down to the bottom?
[0] https://www.postgresql.org/docs/current/routine-vacuuming.html#AUTOVACUUM
--
Sami Imseih
Amazon Web Services
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-27 21:15 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-27 21:15 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Mon, Oct 27, 2025 at 12:47:15PM -0500, Sami Imseih wrote:
> 1/ Should we add documentation explaining this prioritization behavior in [0]?
>
> I wrote a sql that returns the tables and scores, which I found was
> useful when I was testing this out, so having the actually rules spelled out
> in docs will actually be super useful.
Can you elaborate on how it would be useful? I'd be open to adding a short
note that autovacuum attempts to prioritize the tables in a smart way, but
I'm not sure I see the value of documenting every detail. I also don't
want to add too much friction to future changes to the prioritization
logic.
> If we don't want to go that much in depth, at minimum the docs should say:
>
> "Autovacuum prioritizes tables based on how far they exceed their thresholds
> or if they are approaching wraparound limits." so a DBA can understand
> this behavior.
Yeah, I would probably choose to keep it relatively vague like this.
> * The score is calculated as the maximum of the ratios of each of the table's
> * relevant values to its threshold. For example, if the number of inserted
> * tuples is 100, and the insert threshold for the table is 80, the insert
> * score is 1.25.
>
> Should we consider clamping down on the score when
> reltuples = -1, otherwise the scores for such tables ( new tables
> with a large amount of ingested data ) will be over-inflated? Perhaps,
> if reltuples = -1 ( # of reltuples not known ), then give a score of .5,
> so we are not over-prioritizing but not pushing down to the bottom?
I'm not sure it's worth expending too much energy to deal with this. In
the worst case, the table will be given an arbitrarily high priority the
first time it is vacuumed, but AFAICT that's it. But that's already the
case, as the thresholds will be artificially low before the first
VACUUM/ANALYZE.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-27 22:35 ` Sami Imseih <samimseih@gmail.com>
2025-10-27 23:16 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-10-27 22:35 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> > I wrote a sql that returns the tables and scores, which I found was
> > useful when I was testing this out, so having the actually rules spelled out
> > in docs will actually be super useful.
>
> Can you elaborate on how it would be useful? I'd be open to adding a short
> note that autovacuum attempts to prioritize the tables in a smart way, but
> I'm not sure I see the value of documenting every detail.
We discuss the threshold calculations in the documentation, and users
can write scripts to monitor which tables are eligible. However, there
is nothing that indicates which table autovacuum will work on next (I
have been asked that question by users a few times, sometimes out of
curiosity, or because they are monitoring vacuum activity and wondering
when their important table will get a vacuum cycle, or if they should
kick off a manual vacuum). With the scoring system, it will be much more
difficult to explain, unless someone walks through the code.
> I also don't
> want to add too much friction to future changes to the prioritization
> logic.
Maybe future changes is a good reason to document the way autovacuum
prioritizes, since this is a user-facing change.
> > If we don't want to go that much in depth, at minimum the docs should say:
> >
> > "Autovacuum prioritizes tables based on how far they exceed their thresholds
> > or if they are approaching wraparound limits." so a DBA can understand
> > this behavior.
>
> Yeah, I would probably choose to keep it relatively vague like this.
With all the above said, starting with something small is definitely better
than nothing.
> > * The score is calculated as the maximum of the ratios of each of the table's
> > * relevant values to its threshold. For example, if the number of inserted
> > * tuples is 100, and the insert threshold for the table is 80, the insert
> > * score is 1.25.
> >
> > Should we consider clamping down on the score when
> > reltuples = -1, otherwise the scores for such tables ( new tables
> > with a large amount of ingested data ) will be over-inflated? Perhaps,
> > if reltuples = -1 ( # of reltuples not known ), then give a score of .5,
> > so we are not over-prioritizing but not pushing down to the bottom?
>
> I'm not sure it's worth expending too much energy to deal with this. In
> the worst case, the table will be given an arbitrarily high priority the
> first time it is vacuumed, but AFAICT that's it. But that's already the
> case, as the thresholds will be artificially low before the first
> VACUUM/ANALYZE.
I can think of scenarios where they may be workloads that create/drops
staging tables and load some data ( like batch processing ) where this
may become an issue because we are now forcing such tables to the top
of the list, potentially impacting other tables from getting vacuum cycles.
It could happen now, but the difference with this change is we are
forcing these tables to the top of the priority; based on an unknown
value (pg_class.reltuples = -1).
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-27 23:16 ` David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-27 23:16 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, 28 Oct 2025 at 11:35, Sami Imseih <samimseih@gmail.com> wrote:
> We discuss the threshold calculations in the documentation, and users
> can write scripts to monitor which tables are eligible. However, there
> is nothing that indicates which table autovacuum will work on next (I
> have been asked that question by users a few times, sometimes out of
> curiosity, or because they are monitoring vacuum activity and wondering
> when their important table will get a vacuum cycle, or if they should
> kick off a manual vacuum). With the scoring system, it will be much more
> difficult to explain, unless someone walks through the code.
I think it's reasonable to want to document how autovacuum prioritises
tables, but maybe not in too much detail. Longer term, I think it
would be good to have a pg_catalog view for this which showed the
relid or schema/relname, and the output values of
relation_needs_vacanalyze(). If we had that and we documented that
autovacuum workers work from that list, but they just may have an
older snapshot of it, then that might help make the score easier to
document. It would also allow people to question the scores as I
expect at least some people might not agree with the priorities. That
would allow us to consider tuning the score calculation if someone
points out a deficiency with the current calculation.
Also, longer-term, it also doesn't seem that unreasonable that the
autovacuum worker might want to refresh the tables_to_process once it
finishes a table and if autovacuum_naptime * $value units of time have
passed since it was last checked. That would allow the worker to deal
with and react accordingly when scores have changed significantly
since it last checked. I mean, it might be days between when
autovacuum calculates the scores and finally vacuums the table when
the list is long, of it it was tied up with large tables. Other
workers may have gotten to some of the tables too, so the score may
have dropped, but again made its way above the threshold, but to a
lesser extent.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 23:16 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-28 21:06 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-29 15:24 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-28 21:06 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Oct 28, 2025 at 12:16:28PM +1300, David Rowley wrote:
> I think it's reasonable to want to document how autovacuum prioritises
> tables, but maybe not in too much detail. Longer term, I think it
> would be good to have a pg_catalog view for this which showed the
> relid or schema/relname, and the output values of
> relation_needs_vacanalyze(). If we had that and we documented that
> autovacuum workers work from that list, but they just may have an
> older snapshot of it, then that might help make the score easier to
> document. It would also allow people to question the scores as I
> expect at least some people might not agree with the priorities. That
> would allow us to consider tuning the score calculation if someone
> points out a deficiency with the current calculation.
>
> Also, longer-term, it also doesn't seem that unreasonable that the
> autovacuum worker might want to refresh the tables_to_process once it
> finishes a table and if autovacuum_naptime * $value units of time have
> passed since it was last checked. That would allow the worker to deal
> with and react accordingly when scores have changed significantly
> since it last checked. I mean, it might be days between when
> autovacuum calculates the scores and finally vacuums the table when
> the list is long, of it it was tied up with large tables. Other
> workers may have gotten to some of the tables too, so the score may
> have dropped, but again made its way above the threshold, but to a
> lesser extent.
Agreed on both points.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 23:16 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-29 15:24 ` Sami Imseih <samimseih@gmail.com>
2025-10-29 16:07 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-10-29 15:24 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> On Tue, Oct 28, 2025 at 12:16:28PM +1300, David Rowley wrote:
> > I think it's reasonable to want to document how autovacuum prioritises
> > tables, but maybe not in too much detail. Longer term, I think it
> > would be good to have a pg_catalog view for this which showed the
> > relid or schema/relname, and the output values of
> > relation_needs_vacanalyze(). If we had that and we documented that
> > autovacuum workers work from that list, but they just may have an
> > older snapshot of it, then that might help make the score easier to
> > document. It would also allow people to question the scores as I
> > expect at least some people might not agree with the priorities. That
> > would allow us to consider tuning the score calculation if someone
> > points out a deficiency with the current calculation.
> >
> > Also, longer-term, it also doesn't seem that unreasonable that the
> > autovacuum worker might want to refresh the tables_to_process once it
> > finishes a table and if autovacuum_naptime * $value units of time have
> > passed since it was last checked. That would allow the worker to deal
> > with and react accordingly when scores have changed significantly
> > since it last checked. I mean, it might be days between when
> > autovacuum calculates the scores and finally vacuums the table when
> > the list is long, of it it was tied up with large tables. Other
> > workers may have gotten to some of the tables too, so the score may
> > have dropped, but again made its way above the threshold, but to a
> > lesser extent.
>
> Agreed on both points.
I think we do need some documentation about this behavior, which v6 is
still missing.
Another thing I have been contemplating about is the change in prioritization
and the resulting difference in the order in which tables are vacuumed
is what it means for workloads in which autovacuum tuning that was
done with the current assumptions will no longer be beneficial.
Let's imagine staging tables that get created and dropped during
some batch processing window and they see huge data
ingestion/changes. The current scan will make these less of a priority
naturally in relation to other permanent tables, but with the new priority,
we are making these staging tables more of a priority. Users will now
need to maybe turn off autovacuum on a per-table level to prevent this
scenario. That is just one example.
What I am also trying to say is should we provide a way, I hate
to say a GUC, for users to go back to the old behavior? or am I
overstating the risk here?
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 23:16 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-29 15:24 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-29 16:07 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 02:58 ` Re: another autovacuum scheduling thread wenhui qiu <qiuwenhuifx@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-29 16:07 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, Oct 29, 2025 at 10:24:17AM -0500, Sami Imseih wrote:
> I think we do need some documentation about this behavior, which v6 is
> still missing.
Would you be interested in giving that part a try?
> Another thing I have been contemplating about is the change in prioritization
> and the resulting difference in the order in which tables are vacuumed
> is what it means for workloads in which autovacuum tuning that was
> done with the current assumptions will no longer be beneficial.
>
> Let's imagine staging tables that get created and dropped during
> some batch processing window and they see huge data
> ingestion/changes. The current scan will make these less of a priority
> naturally in relation to other permanent tables, but with the new priority,
> we are making these staging tables more of a priority. Users will now
> need to maybe turn off autovacuum on a per-table level to prevent this
> scenario. That is just one example.
>
> What I am also trying to say is should we provide a way, I hate
> to say a GUC, for users to go back to the old behavior? or am I
> overstating the risk here?
It's probably worth testing out this scenario, but I can't say I'm terribly
worried. Those kinds of tables are already getting chosen by autovacuum
earlier due to reltuples == -1, and this patch will just move them to the
front of the list that autovacuum creates. In any case, I'd really like to
avoid a GUC or fallback switch here.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 23:16 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-29 15:24 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 16:07 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-30 02:58 ` wenhui qiu <qiuwenhuifx@gmail.com>
2025-10-30 03:41 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: wenhui qiu @ 2025-10-30 02:58 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
HI Nathan
> That approach would begin aggressively scaling the priority of tables
> sooner, but I don't know if that's strictly better. In any case, I'd like
> to avoid making the score calculation too magical.
In fact, with the introduction of the vacuum_max_eager_freeze_failure_rate
feature, if a table’s age still exceeds more than 1.x times the
autovacuum_freeze_max_age, it suggests that the vacuum freeze process is
not functioning properly. Once the age surpasses vacuum_failsafe_age,
wraparound issues are likely to occur soon.Taking the average of
vacuum_failsafe_age and autovacuum_freeze_max_age is not a complex
approach. Under the default configuration, this average already exceeds
four times the autovacuum_freeze_max_age. At that stage, a DBA should have
already intervened to investigate and resolve why the table age is not
decreasing.
Thanks
On Thu, Oct 30, 2025 at 12:07 AM Nathan Bossart <nathandbossart@gmail.com>
wrote:
> On Wed, Oct 29, 2025 at 10:24:17AM -0500, Sami Imseih wrote:
> > I think we do need some documentation about this behavior, which v6 is
> > still missing.
>
> Would you be interested in giving that part a try?
>
> > Another thing I have been contemplating about is the change in
> prioritization
> > and the resulting difference in the order in which tables are vacuumed
> > is what it means for workloads in which autovacuum tuning that was
> > done with the current assumptions will no longer be beneficial.
> >
> > Let's imagine staging tables that get created and dropped during
> > some batch processing window and they see huge data
> > ingestion/changes. The current scan will make these less of a priority
> > naturally in relation to other permanent tables, but with the new
> priority,
> > we are making these staging tables more of a priority. Users will now
> > need to maybe turn off autovacuum on a per-table level to prevent this
> > scenario. That is just one example.
> >
> > What I am also trying to say is should we provide a way, I hate
> > to say a GUC, for users to go back to the old behavior? or am I
> > overstating the risk here?
>
> It's probably worth testing out this scenario, but I can't say I'm terribly
> worried. Those kinds of tables are already getting chosen by autovacuum
> earlier due to reltuples == -1, and this patch will just move them to the
> front of the list that autovacuum creates. In any case, I'd really like to
> avoid a GUC or fallback switch here.
>
> --
> nathan
>
>
>
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 23:16 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-29 15:24 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 16:07 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 02:58 ` Re: another autovacuum scheduling thread wenhui qiu <qiuwenhuifx@gmail.com>
@ 2025-10-30 03:41 ` David Rowley <dgrowleyml@gmail.com>
2025-10-30 06:48 ` Re: another autovacuum scheduling thread wenhui qiu <qiuwenhuifx@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-30 03:41 UTC (permalink / raw)
To: wenhui qiu <qiuwenhuifx@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, 30 Oct 2025 at 15:58, wenhui qiu <qiuwenhuifx@gmail.com> wrote:
> In fact, with the introduction of the vacuum_max_eager_freeze_failure_rate feature, if a table’s age still exceeds more than 1.x times the autovacuum_freeze_max_age, it suggests that the vacuum freeze process is not functioning properly. Once the age surpasses vacuum_failsafe_age, wraparound issues are likely to occur soon.Taking the average of vacuum_failsafe_age and autovacuum_freeze_max_age is not a complex approach. Under the default configuration, this average already exceeds four times the autovacuum_freeze_max_age. At that stage, a DBA should have already intervened to investigate and resolve why the table age is not decreasing.
I don't think anyone would like to modify PostgreSQL in any way that
increases the chances that a table gets as old as vacuum_failsafe_age.
Regardless of the order in which tables are vacuumed, if a table gets
as old as that then vacuum is configured to run too slowly, or there
are not enough workers configured to cope with the given amount of
work. I think we need to tackle prioritisation and rate limiting as
two separate items. Nathan is proposing to improve the prioritisation
in this thread and it seems to me that your concerns are with rate
limiting. I've suggested an idea that might help with reducing the
cost_delay based on the score of the table in this thread. I'd rather
not introduce that as a topic for further discussion here (I imagine
Nathan agrees). It's not as if the server is going to consume 1
billion xids in 5 mins. It's at least going to take a day to days or
longer for that to happen and if autovacuum has not managed to get on
top of the workload in that time, then it's configured to run too
slowly and the cost_limit or delay needs to be adjusted.
My concern is that there are countless problems with autovacuum and if
you try and lump them all into a single thread to fix them all at
once, we'll get nowhere. Autovacuum was added to core in 8.1, 20 years
ago and I don't believe we've done anything to change the ratelimiting
aside from reducing the default cost_delay since then. It'd be good to
fix that at some point, just not here, please.
FWIW, I agree with Nathan about keeping the score calculation
non-magical. The score should be simple and easy to document. We can
introduce complexity to it as and when it's needed and when the
supporting evidence arrives, rather than from people waving their
hands.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 23:16 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-29 15:24 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 16:07 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 02:58 ` Re: another autovacuum scheduling thread wenhui qiu <qiuwenhuifx@gmail.com>
2025-10-30 03:41 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-30 06:48 ` wenhui qiu <qiuwenhuifx@gmail.com>
2025-10-30 10:36 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: wenhui qiu @ 2025-10-30 06:48 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
HI
I think there might be some misunderstanding — I’m only suggesting
changing
effective_xid_failsafe_age = Max(vacuum_failsafe_age,
autovacuum_freeze_max_age * 1.05);
to
effective_xid_failsafe_age = (vacuum_failsafe_age +
autovacuum_freeze_max_age) / 2.0;
In the current logic, effective_xid_failsafe_age is almost always equal to
vacuum_failsafe_age.
As a result, increasing the vacuum priority only when a table’s age reaches
vacuum_failsafe_age is too late.
Thanks
On Thu, Oct 30, 2025 at 11:42 AM David Rowley <dgrowleyml@gmail.com> wrote:
> On Thu, 30 Oct 2025 at 15:58, wenhui qiu <qiuwenhuifx@gmail.com> wrote:
> > In fact, with the introduction of the
> vacuum_max_eager_freeze_failure_rate feature, if a table’s age still
> exceeds more than 1.x times the autovacuum_freeze_max_age, it suggests that
> the vacuum freeze process is not functioning properly. Once the age
> surpasses vacuum_failsafe_age, wraparound issues are likely to occur
> soon.Taking the average of vacuum_failsafe_age and
> autovacuum_freeze_max_age is not a complex approach. Under the default
> configuration, this average already exceeds four times the
> autovacuum_freeze_max_age. At that stage, a DBA should have already
> intervened to investigate and resolve why the table age is not decreasing.
>
> I don't think anyone would like to modify PostgreSQL in any way that
> increases the chances that a table gets as old as vacuum_failsafe_age.
> Regardless of the order in which tables are vacuumed, if a table gets
> as old as that then vacuum is configured to run too slowly, or there
> are not enough workers configured to cope with the given amount of
> work. I think we need to tackle prioritisation and rate limiting as
> two separate items. Nathan is proposing to improve the prioritisation
> in this thread and it seems to me that your concerns are with rate
> limiting. I've suggested an idea that might help with reducing the
> cost_delay based on the score of the table in this thread. I'd rather
> not introduce that as a topic for further discussion here (I imagine
> Nathan agrees). It's not as if the server is going to consume 1
> billion xids in 5 mins. It's at least going to take a day to days or
> longer for that to happen and if autovacuum has not managed to get on
> top of the workload in that time, then it's configured to run too
> slowly and the cost_limit or delay needs to be adjusted.
>
> My concern is that there are countless problems with autovacuum and if
> you try and lump them all into a single thread to fix them all at
> once, we'll get nowhere. Autovacuum was added to core in 8.1, 20 years
> ago and I don't believe we've done anything to change the ratelimiting
> aside from reducing the default cost_delay since then. It'd be good to
> fix that at some point, just not here, please.
>
> FWIW, I agree with Nathan about keeping the score calculation
> non-magical. The score should be simple and easy to document. We can
> introduce complexity to it as and when it's needed and when the
> supporting evidence arrives, rather than from people waving their
> hands.
>
> David
>
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-27 23:16 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-29 15:24 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 16:07 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 02:58 ` Re: another autovacuum scheduling thread wenhui qiu <qiuwenhuifx@gmail.com>
2025-10-30 03:41 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-30 06:48 ` Re: another autovacuum scheduling thread wenhui qiu <qiuwenhuifx@gmail.com>
@ 2025-10-30 10:36 ` David Rowley <dgrowleyml@gmail.com>
0 siblings, 0 replies; 150+ messages in thread
From: David Rowley @ 2025-10-30 10:36 UTC (permalink / raw)
To: wenhui qiu <qiuwenhuifx@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, 30 Oct 2025 at 19:48, wenhui qiu <qiuwenhuifx@gmail.com> wrote:
> I think there might be some misunderstanding — I’m only suggesting changing
> effective_xid_failsafe_age = Max(vacuum_failsafe_age,
> autovacuum_freeze_max_age * 1.05);
> to
> effective_xid_failsafe_age = (vacuum_failsafe_age + autovacuum_freeze_max_age) / 2.0;
> In the current logic, effective_xid_failsafe_age is almost always equal to vacuum_failsafe_age.
> As a result, increasing the vacuum priority only when a table’s age reaches vacuum_failsafe_age is too late.
I understand your proposal. The autovacuum will trigger for the
wraparound at autovacuum_freeze_max_age, so for autovacuum still not
to have gotten to the table by the time the table is aged at
vacuum_failsafe_age, it means autovacuum isn't working quickly enough
to get through the workload, therefore the problem is with the speed
of autovacuum not the priority of autovacuum.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-27 22:47 ` David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-10-27 22:47 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
The patch is starting to look good. Here's a review of v5:
1. I think the following code at the bottom of
relation_needs_vacanalyze() can be deleted. You've added the check to
ensure *doanalyze never gets set to true for pg_statistic.
/* ANALYZE refuses to work with pg_statistic */
if (relid == StatisticRelationId)
*doanalyze = false;
2. As #1, but in recheck_relation_needs_vacanalyze(), the following I
think can now be removed:
/* ignore ANALYZE for toast tables */
if (classForm->relkind == RELKIND_TOASTVALUE)
*doanalyze = false;
3. Would you be able to include what the idea behind the * 1.05 in the
preceding comment?
On Tue, 28 Oct 2025 at 05:06, Nathan Bossart <nathandbossart@gmail.com> wrote:
> + effective_xid_failsafe_age = Max(vacuum_failsafe_age,
> + autovacuum_freeze_max_age * 1.05);
> + effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
> + autovacuum_multixact_freeze_max_age * 1.05);
I assume it's to workaround some strange configuration settings, but
don't know for sure, or why 1.05 is a good value.
4. I think it might be neater to format the following as 3 separate "if" tests:
> + if (force_vacuum ||
> + vactuples > vacthresh ||
> + (vac_ins_base_thresh >= 0 && instuples > vacinsthresh))
> + {
> + *dovacuum = true;
> + *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
> + if (vac_ins_base_thresh >= 0)
> + *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
> + }
> + else
> + *dovacuum = false;
i.e:
if (force_vacuum)
*dovacuum = true;
if (vactuples > vacthresh)
{
*dovacuum = true;
*score = Max(*score, (double) vactuples / Max(vacthresh, 1));
}
if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
{
*dovacuum = true;
*score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
}
and also get rid of all the "else *dovacuum = false;" (and *dovacuum =
false) in favour of setting those to false at the top of the function.
It's just getting harder to track that those parameters are getting
set in all cases when they're meant to be.
doing that also gets rid of the duplicative "if (vac_ins_base_thresh
>= 0)" check and also saves doing the score calc when the inputs to it
don't make sense. The current code is relying on Max always picking
the current *score when the threshold isn't met.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-28 21:03 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-28 21:03 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Oct 28, 2025 at 11:47:08AM +1300, David Rowley wrote:
> 1. I think the following code at the bottom of
> relation_needs_vacanalyze() can be deleted. You've added the check to
> ensure *doanalyze never gets set to true for pg_statistic.
>
> /* ANALYZE refuses to work with pg_statistic */
> if (relid == StatisticRelationId)
> *doanalyze = false;
>
> 2. As #1, but in recheck_relation_needs_vacanalyze(), the following I
> think can now be removed:
>
> /* ignore ANALYZE for toast tables */
> if (classForm->relkind == RELKIND_TOASTVALUE)
> *doanalyze = false;
Removed.
> 3. Would you be able to include what the idea behind the * 1.05 in the
> preceding comment?
>
> On Tue, 28 Oct 2025 at 05:06, Nathan Bossart <nathandbossart@gmail.com> wrote:
>> + effective_xid_failsafe_age = Max(vacuum_failsafe_age,
>> + autovacuum_freeze_max_age * 1.05);
>> + effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
>> + autovacuum_multixact_freeze_max_age * 1.05);
>
> I assume it's to workaround some strange configuration settings, but
> don't know for sure, or why 1.05 is a good value.
This is lifted from vacuum_xid_failsafe_check(). As noted in the docs, the
failsafe settings are silently limited to 105% of *_freeze_max_age. I
expanded on this in the comment atop these lines.
> 4. I think it might be neater to format the following as 3 separate "if" tests:
>
>> + if (force_vacuum ||
>> + vactuples > vacthresh ||
>> + (vac_ins_base_thresh >= 0 && instuples > vacinsthresh))
>> + {
>> + *dovacuum = true;
>> + *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
>> + if (vac_ins_base_thresh >= 0)
>> + *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
>> + }
>> + else
>> + *dovacuum = false;
>
> i.e:
>
> if (force_vacuum)
> *dovacuum = true;
>
> if (vactuples > vacthresh)
> {
> *dovacuum = true;
> *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
> }
>
> if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
> {
> *dovacuum = true;
> *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
> }
>
> and also get rid of all the "else *dovacuum = false;" (and *dovacuum =
> false) in favour of setting those to false at the top of the function.
> It's just getting harder to track that those parameters are getting
> set in all cases when they're meant to be.
>
> doing that also gets rid of the duplicative "if (vac_ins_base_thresh
> >= 0)" check and also saves doing the score calc when the inputs to it
> don't make sense. The current code is relying on Max always picking
> the current *score when the threshold isn't met.
Done.
--
nathan
From e77ee5889dbb1e0ff21ea1dfa259b42e8e9111b5 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v6 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 197 ++++++++++++++++++++++------
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 158 insertions(+), 40 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..cff202b9d46 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,15 +2979,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2990,6 +3025,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3000,7 +3061,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,11 +3091,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3081,32 +3148,73 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3169,25 +3277,34 @@ relation_needs_vacanalyze(Oid relid,
NameStr(classForm->relname),
vactuples, vacthresh, anltuples, anlthresh);
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
/*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
*/
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
+ if (vactuples > vacthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ *dovacuum = true;
+ *score = *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
+ }
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ *doanalyze = true;
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ }
+ }
}
/*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bb4e1b37005..6606e6808d3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3007,6 +3007,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v6-0001-autovacuum-scheduling-improvements.patch (13.1K, ../../aQEvm40W3aVizp5Q@nathan/2-v6-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From e77ee5889dbb1e0ff21ea1dfa259b42e8e9111b5 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v6 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 197 ++++++++++++++++++++++------
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 158 insertions(+), 40 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..cff202b9d46 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,15 +2979,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2990,6 +3025,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3000,7 +3061,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,11 +3091,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3081,32 +3148,73 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3169,25 +3277,34 @@ relation_needs_vacanalyze(Oid relid,
NameStr(classForm->relname),
vactuples, vacthresh, anltuples, anlthresh);
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
/*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
*/
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
+ if (vactuples > vacthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ *dovacuum = true;
+ *score = *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
+ }
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ *doanalyze = true;
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ }
+ }
}
/*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bb4e1b37005..6606e6808d3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3007,6 +3007,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-28 22:44 ` Sami Imseih <samimseih@gmail.com>
2025-10-29 03:10 ` Re: another autovacuum scheduling thread wenhui qiu <qiuwenhuifx@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Sami Imseih @ 2025-10-28 22:44 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> Done.
My compiler is complaining about v6
"../src/backend/postmaster/autovacuum.c:3293:32: warning: operation on
‘*score’ may be undefined [-Wsequence-point]
3293 | *score = *score = Max(*score, (double)
instuples / Max(vacinsthresh, 1));
[2/2] Linking target src/backend/postgres"
shouldn't just be like below?
*score =Max(*score, (double) instuples / Max(vacinsthresh, 1));
--
Sami
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-29 03:10 ` wenhui qiu <qiuwenhuifx@gmail.com>
2025-10-29 15:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: wenhui qiu @ 2025-10-29 03:10 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
HI Nathan Bossart
> + if (vactuples > vacthresh)
> + {
> + *dovacuum = true;
> + *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
> + }
> +
> + if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
> + {
> + *dovacuum = true;
> + *score = *score = Max(*score, (double) instuples / Max(vacinsthresh,
1));
> + }
I think it ( *score = *score = Max(*score, (double) instuples /
Max(vacinsthresh, 1));) I believe this must be a slip of the hand on your
part, having copied an extra one.
I also suggest add debug log for score
ereport(DEBUG2,
(errmsg("autovacuum candidate: %s (score=%.3f)",
get_rel_name(table->oid), table->score)));
> + effective_xid_failsafe_age = Max(vacuum_failsafe_age,
> + autovacuum_freeze_max_age * 1.05);
Typically, DBAs avoid setting autovacuum_freeze_max_age too close to
vacuum_failsafe_age. Therefore, your logic most likely uses the
vacuum_failsafe_age value.
Would taking the average of the two be a better approach?
#
root@localhost:/data/pgsql/pg18data# grep vacuum_failsafe_age
postgresql.conf
#vacuum_failsafe_age = 1600000000
root@localhost:/data/pgsql/pg18data# grep autovacuum_freeze_max_age
postgresql.conf
#autovacuum_freeze_max_age = 200000000 # maximum XID age before forced
vacuum
Thanks
On Wed, Oct 29, 2025 at 6:45 AM Sami Imseih <samimseih@gmail.com> wrote:
> > Done.
>
> My compiler is complaining about v6
>
> "../src/backend/postmaster/autovacuum.c:3293:32: warning: operation on
> ‘*score’ may be undefined [-Wsequence-point]
> 3293 | *score = *score = Max(*score, (double)
> instuples / Max(vacinsthresh, 1));
> [2/2] Linking target src/backend/postgres"
>
> shouldn't just be like below?
>
> *score =Max(*score, (double) instuples / Max(vacinsthresh, 1));
>
>
> --
> Sami
>
>
>
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 03:10 ` Re: another autovacuum scheduling thread wenhui qiu <qiuwenhuifx@gmail.com>
@ 2025-10-29 15:58 ` Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 0 replies; 150+ messages in thread
From: Nathan Bossart @ 2025-10-29 15:58 UTC (permalink / raw)
To: wenhui qiu <qiuwenhuifx@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, Oct 29, 2025 at 11:10:55AM +0800, wenhui qiu wrote:
> Typically, DBAs avoid setting autovacuum_freeze_max_age too close to
> vacuum_failsafe_age. Therefore, your logic most likely uses the
> vacuum_failsafe_age value.
> Would taking the average of the two be a better approach?
That approach would begin aggressively scaling the priority of tables
sooner, but I don't know if that's strictly better. In any case, I'd like
to avoid making the score calculation too magical.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-29 15:51 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-29 15:51 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Oct 28, 2025 at 05:44:37PM -0500, Sami Imseih wrote:
> My compiler is complaining about v6
>
> "../src/backend/postmaster/autovacuum.c:3293:32: warning: operation on
> ‘*score’ may be undefined [-Wsequence-point]
> 3293 | *score = *score = Max(*score, (double)
> instuples / Max(vacinsthresh, 1));
> [2/2] Linking target src/backend/postgres"
>
> shouldn't just be like below?
>
> *score =Max(*score, (double) instuples / Max(vacinsthresh, 1));
Oops. I fixed that typo in v7.
--
nathan
From a225d5965286c87403bdcad2aaad0ec5def7e115 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v7 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 197 ++++++++++++++++++++++------
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 158 insertions(+), 40 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..e48bb06253b 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,15 +2979,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2990,6 +3025,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3000,7 +3061,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,11 +3091,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3081,32 +3148,73 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3169,25 +3277,34 @@ relation_needs_vacanalyze(Oid relid,
NameStr(classForm->relname),
vactuples, vacthresh, anltuples, anlthresh);
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
/*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
*/
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
+ if (vactuples > vacthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
+ }
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ *doanalyze = true;
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ }
+ }
}
/*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index ac2da4c98cf..bb977229f75 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3007,6 +3007,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v7-0001-autovacuum-scheduling-improvements.patch (13.1K, ../../aQI39ln_jZ8qorLE@nathan/2-v7-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From a225d5965286c87403bdcad2aaad0ec5def7e115 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v7 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 197 ++++++++++++++++++++++------
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 158 insertions(+), 40 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 5084af7dfb6..e48bb06253b 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -97,6 +97,7 @@
#include "storage/procsignal.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
+#include "utils/float.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
#include "utils/guc_hooks.h"
@@ -310,6 +311,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +358,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1897,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return float8_cmp_internal(t2->score, t1->score);
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1919,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2031,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2072,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2140,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2173,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2934,6 +2971,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2941,15 +2979,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2990,6 +3025,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3000,7 +3061,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3029,11 +3091,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3081,32 +3148,73 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3169,25 +3277,34 @@ relation_needs_vacanalyze(Oid relid,
NameStr(classForm->relname),
vactuples, vacthresh, anltuples, anlthresh);
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
/*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
*/
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
+ if (vactuples > vacthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
+ }
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ *doanalyze = true;
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ }
+ }
}
/*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index ac2da4c98cf..bb977229f75 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3007,6 +3007,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-30 20:05 ` Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Robert Haas @ 2025-10-30 20:05 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, Oct 29, 2025 at 11:51 AM Nathan Bossart
<nathandbossart@gmail.com> wrote:
> Oops. I fixed that typo in v7.
Are you planning to do some practical experimentation with this? I
feel like it would be a good idea to set up some kind of a test case
where this is expected to provide a benefit and see if it actually
does; and also maybe set up a test case where it will reorder the
tables but with no practical difference in the outcome expected and
verify that, in fact, nothing changes.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-10-30 21:02 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-30 21:02 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Oct 30, 2025 at 04:05:19PM -0400, Robert Haas wrote:
> Are you planning to do some practical experimentation with this? I
> feel like it would be a good idea to set up some kind of a test case
> where this is expected to provide a benefit and see if it actually
> does; and also maybe set up a test case where it will reorder the
> tables but with no practical difference in the outcome expected and
> verify that, in fact, nothing changes.
Yes. I've been thinking through how I want to test this but have yet to
actually do so. If you have ideas, I'm all ears.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-30 21:05 ` Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-10-30 21:05 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> On Thu, Oct 30, 2025 at 04:05:19PM -0400, Robert Haas wrote:
> > Are you planning to do some practical experimentation with this? I
> > feel like it would be a good idea to set up some kind of a test case
> > where this is expected to provide a benefit and see if it actually
> > does; and also maybe set up a test case where it will reorder the
> > tables but with no practical difference in the outcome expected and
> > verify that, in fact, nothing changes.
>
> Yes. I've been thinking through how I want to test this but have yet to
> actually do so. If you have ideas, I'm all ears.
FWIW, I've been putting some scripts together to test some workloads
and I will share shortly what I have.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-31 00:38 ` Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-10-31 00:38 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> FWIW, I've been putting some scripts together to test some workloads
> and I will share shortly what I have.
Here is my attempt to test the behavior with the new prioritization.
I wanted a way to run the same tests with different workloads, both with
and without the prioritization patch, and to see if anything stands out as
suspicious in terms of autovacuum or autoanalyze activity. For example,
certain tables showing too little or too much autovacuum activity.
The scripts I put together (attached) run a busy update workload (OLTP)
and a separate batch workload. They use pgbench to execute custom scripts
that are generated on the fly.
The results are summarized by the average number of autovacuum and
autoanalyze runs *per table*, along with some other DML activity stats to
ensure that the workloads being compared have similar DML activity.
Using the scripts:
Place the attached scripts in a specific directory, and modify the
section under "Caller should adjust these values" in run_workloads.sh
to adjust the workload. The scripts assume you have a running cluster with
your specific config file adjusted for the test.
Once ready, call run_workloads.sh and at the end a summary will show up
as you see below. Hopefully it works for you :)
The summary.sh script can also be run while the workloads are executing.
Here is a example of a test I wanted to run based on the discussion [0]:
This scenario is one that was mentioned, but there are others in which a
batch process performing inserts only is prioritized over the update
workload.
I ran this test for 10 minutes, using 200 clients for the update workload
and 5 clients for the batch workload, with the following configuration:
```
max_connections=1000;
autovacuum_naptime = '10s'
shared_buffers = '4GB'
autovacuum_max_workers = 6
```
-- HEAD
```
Total Activity
-[ RECORD 1 ]-------------+----------
total_n_dead_tup | 985183
total_n_mod_since_analyze | 220294866
total_reltuples | 247690373
total_autovacuum_count | 137
total_autoanalyze_count | 470
total_n_tup_upd | 7720012
total_n_tup_ins | 446683000
table_count | 105
Activity By Workload Type
-[ RECORD 1 ]-----------------+----------------
table_group | batch_tables
** avg_autovacuum_count | 7.400
** avg_autoanalyze_count | 8.000
avg_vacuum_count | 0.000
avg_analyze_count | 0.000
rows_inserted | 436683000
rows_updated | 0
rows_hot_updated | 0
table_count | 5
-[ RECORD 2 ]-----------------+----------------
table_group | numbered_tables
** avg_autovacuum_count | 1.000
** avg_autoanalyze_count | 4.300
avg_vacuum_count | 1.000
avg_analyze_count | 0.000
rows_inserted | 10000000
rows_updated | 7720012
rows_hot_updated | 7094573
table_count | 100
```
-- with v7 applied
```
Total Activity
-[ RECORD 1 ]-------------+----------
total_n_dead_tup | 1233045
total_n_mod_since_analyze | 137843507
total_reltuples | 350704437
total_autovacuum_count | 146
total_autoanalyze_count | 605
total_n_tup_upd | 7896354
total_n_tup_ins | 487974000
table_count | 105
Activity By Workload Type
-[ RECORD 1 ]-----------------+----------------
table_group | batch_tables
** avg_autovacuum_count | 11.000
** avg_autoanalyze_count | 13.200
avg_vacuum_count | 0.000
avg_analyze_count | 0.000
rows_inserted | 477974000
rows_updated | 0
rows_hot_updated | 0
table_count | 5
-[ RECORD 2 ]-----------------+----------------
table_group | numbered_tables
** avg_autovacuum_count | 0.910
** avg_autoanalyze_count | 5.390
avg_vacuum_count | 1.000
avg_analyze_count | 0.000
rows_inserted | 10000000
rows_updated | 7896354
rows_hot_updated | 7123134
table_count | 100
```
The results above show what I expected: the batch tables receive higher
priority, as seen from the averages of autovacuum and autoanalyze runs.
This behavior is expected, but it may catch some users by surprise after
an upgrade, since certain tables will now receive more attention than
others. Longer tests might also show more bloat accumulating on heavily
updated tables. In such cases, a user may need to adjust autovacuum
settings on a per-table basis to restore the previous behavior.
So, I am not quite sure what is the best way to test except for trying
to find these non steady state workloads and see the impact of the
prioritization change to (auto)vacuum/analyze activity .
Maybe there is a better way?
[0] https://www.postgresql.org/message-id/aQI7tGEs8IOPxG64%40nathan
--
Sami Imseih
Amazon Web Services (AWS)
Attachments:
[application/x-sh] batch.sh (1.4K, ../../CAA5RZ0tpV3PRHejTGG5-LSsqNKKV0qP=SDvurs8wj7pTk7jYJw@mail.gmail.com/2-batch.sh)
download
[application/x-sh] summary.sh (1.5K, ../../CAA5RZ0tpV3PRHejTGG5-LSsqNKKV0qP=SDvurs8wj7pTk7jYJw@mail.gmail.com/3-summary.sh)
download
[application/x-sh] oltp.sh (3.1K, ../../CAA5RZ0tpV3PRHejTGG5-LSsqNKKV0qP=SDvurs8wj7pTk7jYJw@mail.gmail.com/4-oltp.sh)
download
[application/x-sh] run_workloads.sh (3.3K, ../../CAA5RZ0tpV3PRHejTGG5-LSsqNKKV0qP=SDvurs8wj7pTk7jYJw@mail.gmail.com/5-run_workloads.sh)
download
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-10-31 20:12 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-31 20:12 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Oct 30, 2025 at 07:38:15PM -0500, Sami Imseih wrote:
> Here is my attempt to test the behavior with the new prioritization.
Thanks.
> The results above show what I expected: the batch tables receive higher
> priority, as seen from the averages of autovacuum and autoanalyze runs.
> This behavior is expected, but it may catch some users by surprise after
> an upgrade, since certain tables will now receive more attention than
> others. Longer tests might also show more bloat accumulating on heavily
> updated tables. In such cases, a user may need to adjust autovacuum
> settings on a per-table basis to restore the previous behavior.
Interesting. From these results, it almost sounds as if we're further
amplifying the intended effect of commit 06eae9e. That could be a good
thing. Something else I'm curious about is datfrozenxid, i.e., whether
prioritization keeps the database (M)XID ages lower.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-11-01 01:50 ` David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-11-01 01:50 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sat, 1 Nov 2025 at 09:12, Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> On Thu, Oct 30, 2025 at 07:38:15PM -0500, Sami Imseih wrote:
> > The results above show what I expected: the batch tables receive higher
> > priority, as seen from the averages of autovacuum and autoanalyze runs.
> > This behavior is expected, but it may catch some users by surprise after
> > an upgrade, since certain tables will now receive more attention than
> > others. Longer tests might also show more bloat accumulating on heavily
> > updated tables. In such cases, a user may need to adjust autovacuum
> > settings on a per-table basis to restore the previous behavior.
>
> Interesting. From these results, it almost sounds as if we're further
> amplifying the intended effect of commit 06eae9e. That could be a good
> thing. Something else I'm curious about is datfrozenxid, i.e., whether
> prioritization keeps the database (M)XID ages lower.
I wonder if it would be more realistic to throttle the work simulation
to a certain speed with pgbench -R rather than having it go flat out.
The results show that quite a bit higher "rows_inserted" for the
batch_tables with the patched version. Sami didn't mention any changes
to vacuum_cost_limit, so I suspect that autovacuum would be getting
quite behind on this run, which isn't ideal. Rate limiting to
something that the given vacuum_cost_limit could keep up with seems
more realistic. The fact that the patched version did more insert work
in the batch tables does seem a bit unfair as that gave autovacuum
more work to do in the patched test run which would result in the
lower-scoring tables being neglected more in the patched version.
This makes me wonder if we should log the score of the table when the
autovacuum starts for the table. We do calculate the score again in
recheck_relation_needs_vacanalyze() just before doing the
vacuum/analyze, so maybe the score can be stored in the autovac_table
struct and displayed somewhere. Maybe along with the
log_autovacuum_min_duration / log_autoanalyze_min_duration would be
useful. It might be good in there for DBA analysis to give some
visibility on how bad things got before autovacuum got around to
working on a given table.
If we logged the score, we could do the "unpatched" test with the
patched code, just with commenting out the
list_sort(tables_to_process, TableToProcessComparator); It'd then be
interesting to zero the log_auto*_min_duration settings and review the
order differences and how high the scores got. Would the average score
be higher or lower with patched version? I'd guess lower since the
higher scoring tables would tend to get vacuumed later with the
unpatched version and their score would be even higher by the time
autovacuum got to them. I think if the average score has gone down at
the point that the vacuum starts, then that's a very good thing. Maybe
we'd need to write a patch to recalculate the "tables_to_process" List
after a table is vacuumed and autovacuum_naptime has elapsed for us to
see this, else the priorities might have become too outdated. I'd
expect that to be even more true when vacuum_cost_limit is configured
too low.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-01 03:29 ` David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-11-01 03:29 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sat, 1 Nov 2025 at 14:50, David Rowley <dgrowleyml@gmail.com> wrote:
> If we logged the score, we could do the "unpatched" test with the
> patched code, just with commenting out the
> list_sort(tables_to_process, TableToProcessComparator); It'd then be
> interesting to zero the log_auto*_min_duration settings and review the
> order differences and how high the scores got. Would the average score
> be higher or lower with patched version? I'd guess lower since the
> higher scoring tables would tend to get vacuumed later with the
> unpatched version and their score would be even higher by the time
> autovacuum got to them. I think if the average score has gone down at
> the point that the vacuum starts, then that's a very good thing. Maybe
> we'd need to write a patch to recalculate the "tables_to_process" List
> after a table is vacuumed and autovacuum_naptime has elapsed for us to
> see this, else the priorities might have become too outdated. I'd
> expect that to be even more true when vacuum_cost_limit is configured
> too low.
I'm not yet sure how meaningful it is, but I tried adding the
following to recheck_relation_needs_vacanalyze():
elog(LOG, "Performing autovacuum of table \"%s\" with score = %f",
get_rel_name(relid), score);
then after grepping the logs and loading the data into a table and performing:
select case patched when true then 'v7' else 'master' end as
patched,case when left(tab, 11) = 'table_batch' then 'table_batch_*'
when left(tab,6) = 'table_' then 'table_*' else 'other' end,
avg(score) as avg_Score,count(*) as count from autovac where score>0
and score<2000 group by rollup(1,2) order by 2,1;
with vacuum_cost_limit = 5000, I got:
patched | case | avg_score | count
---------+---------------+--------------------+-------
master | other | 2.004997014705882 | 68
v7 | other | 1.9668087323943668 | 71
master | table_* | 1.196698981375357 | 1396
v7 | table_* | 1.2134741693430646 | 1370
master | table_batch_* | 2.1887380086206902 | 116
v7 | table_batch_* | 1.8882025693430664 | 137
master | | 1.3043197367088595 | 1580
v7 | | 1.3059485323193893 | 1578
| | 1.3051336187460454 | 3158
It would still be good to do the rate limiting as there's more work
being done in the patched version. Seems to be about 1.1% more rows in
batch_tables and 0.48% more updates in the numbered_tables in the
patched version.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-06 22:21 ` Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-11-06 22:21 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Thanks for the ideas on improving the test!
I am still trying to see how useful this type of testing is,
but I will share what I have done.
> I wonder if it would be more realistic to throttle the work simulation
> to a certain speed with pgbench -R rather than having it go flat out.
good point
> > If we logged the score, we could do the "unpatched" test with the
> > patched code, just with commenting out the
> > list_sort(tables_to_process, TableToProcessComparator); It'd then be
> > interesting to zero the log_auto*_min_duration settings and review the
> > order differences and how high the scores got. Would the average score
> > be higher or lower with patched version?
I agree. I attached a patch on top of v7 that implements a debug GUC
to enable or disable sorting for testing purposes.
> I'm not yet sure how meaningful it is, but I tried adding the
> following to recheck_relation_needs_vacanalyze():
>
> elog(LOG, "Performing autovacuum of table \"%s\" with score = %f",
> get_rel_name(relid), score);
The same attached patch also implements this log.
I also spent more time working on the test script. I cleaned it up and
combined it into a single script. I added a few things:
- Ability to run with or without the batch workload.
- OLTP tables are no longer the same size; they are created with
different row counts using a minimum and maximum row count and a
multiplier for scaling the next table.
- A background collector for pg_stat_all_tables on relevant tables,
stored in relstats_monitor.log.
- Logs are saved after the run for further analysis, such as examining
the scores.
Also attached is analysis for a run with 16 OLTP tables and 3 batch tables.
It shows that with sorting enabled or disabled, the vacuum/analyze activity
does not show any major differences. OLTP had very similar DML and
autovacuum/autoanalyze activity. A few points to highlight:
1/ In the sorted run, we had an equal number of autovacuums/autoanalyze
on the smaller OLTP tables, as if every eligible table needed both
autovacuum and autoanalyze. The unsorted run was less consistent on
the smaller tables. I observed this on several runs. I don't think it's a big
deal, but interesting nonetheless.
2/ Batch tables in the sorted run had less autovacuum time (1,257,821 vs
962,794 ms), but very similar autovacuum counts.
3/ OLTP tables, on the other hand, had more autovacuum time in the
sorted run (3,590,964 vs 3,852,460 ms), but I do not see much difference
in autovacuum/autoanalyze counts.
Other tests I plan on running:
- batch updates/deletes, since the current batch option only tests append-only
tables.
- OLTP only test.
Also, I am thinking about another sorting strategy based on average
autovacuum/autoanalyze time per table. The idea is to sort ascending by
the greater of the two averages, so workers process quicker tables first
instead of all workers potentially getting hung on the slowest tables.
We can calculate the average now that v18 includes total_autovacuum_time
and total_autoanalyze time.
The way I see it, regardless of prioritization, a few large tables may
still monopolize autovacuum workers. But at least this way, the quick tables
get a chance to get processed first. Will this be an idea worth testing out?
--
Sami Imseih
Amazon Web Services (AWS)
From c24eeeb074acfb790fe44b60b2177482b9afe3c3 Mon Sep 17 00:00:00 2001
From: Ubuntu <ubuntu@ip-172-31-46-230.ec2.internal>
Date: Tue, 4 Nov 2025 15:55:40 +0000
Subject: [PATCH 1/1] autovacuum score logging and sort enable/disable
---
src/backend/postmaster/autovacuum.c | 7 ++++++-
src/backend/utils/misc/guc_parameters.dat | 10 ++++++++++
src/backend/utils/misc/guc_tables.c | 6 ++++++
src/include/postmaster/autovacuum.h | 8 ++++++++
4 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index e48bb06253b..ca9c5c615dc 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -333,6 +333,8 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
+int debug_autovacuum_sort = DEBUG_AUTOVACUUM_SORT_ON;
+
static Oid do_start_worker(void);
static void ProcessAutoVacLauncherInterrupts(void);
pg_noreturn static void AutoVacLauncherShutdown(void);
@@ -2083,6 +2085,7 @@ do_autovacuum(void)
table->oid = relid;
table->score = score;
+ elog(LOG, "adding table:%s,%lf,av=%d,aa=%d", get_rel_name(table->oid), score, dovacuum, doanalyze);
tables_to_process = lappend(tables_to_process, table);
}
@@ -2184,6 +2187,7 @@ do_autovacuum(void)
table->oid = relid;
table->score = score;
+ elog(LOG, "adding table:%s,%lf,av=1,aa=0", get_rel_name(table->oid), score);
tables_to_process = lappend(tables_to_process, table);
}
@@ -2309,7 +2313,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
- list_sort(tables_to_process, TableToProcessComparator);
+ if (debug_autovacuum_sort == DEBUG_AUTOVACUUM_SORT_ON)
+ list_sort(tables_to_process, TableToProcessComparator);
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index d6fc8333850..2bf9ce4ed27 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -3445,6 +3445,16 @@
options => 'debug_parallel_query_options',
},
+{ name => 'debug_autovacuum_sort', type => 'enum', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+ short_desc => 'Enables/Disables the autovacuum sort of eligible tables.',
+ long_desc => 'This can be useful for testing the effect of sorting eligible tables in autovacuum.',
+ flags => 'GUC_NOT_IN_SAMPLE | GUC_EXPLAIN',
+ variable => 'debug_autovacuum_sort',
+ boot_val => 'DEBUG_AUTOVACUUM_SORT_ON',
+ options => 'debug_autovacuum_sort_options',
+},
+
+
{ name => 'password_encryption', type => 'enum', context => 'PGC_USERSET', group => 'CONN_AUTH_AUTH',
short_desc => 'Chooses the algorithm for encrypting passwords.',
variable => 'Password_encryption',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 00c8376cf4d..aaa93d35187 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -404,6 +404,12 @@ static const struct config_enum_entry debug_parallel_query_options[] = {
{NULL, 0, false}
};
+static const struct config_enum_entry debug_autovacuum_sort_options[] = {
+ {"off", DEBUG_AUTOVACUUM_SORT_OFF, false},
+ {"on", DEBUG_AUTOVACUUM_SORT_ON, false},
+ {NULL, 0, false}
+};
+
static const struct config_enum_entry plan_cache_mode_options[] = {
{"auto", PLAN_CACHE_MODE_AUTO, false},
{"force_generic_plan", PLAN_CACHE_MODE_FORCE_GENERIC_PLAN, false},
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 023ac6d5fa8..32688784a06 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -25,6 +25,14 @@ typedef enum
AVW_BRINSummarizeRange,
} AutoVacuumWorkItemType;
+/* possible values for debug_autovacuum_sort */
+typedef enum
+{
+ DEBUG_AUTOVACUUM_SORT_OFF,
+ DEBUG_AUTOVACUUM_SORT_ON,
+} DebugAutovacuumSortMode;
+
+extern PGDLLIMPORT int debug_autovacuum_sort;
/* GUC variables */
extern PGDLLIMPORT bool autovacuum_start_daemon;
--
2.43.0
################################################################
## SORT OFF
################################################################
### from config
BASE_DIR=$HOME/test_autovacuum_prioritization
OLTP_TABLES=16
OLTP_ROWS_START=1000
OLTP_ROWS_MULTIPLIER=2
OLTP_MAX_ROWS=5000000
BATCH_TABLES=3
BATCH_SIZE=100000
BATCH_CONNECTIONS=5
OLTP_CONNECTIONS=200
TIMEOUT=1800
OLTP_RATE=15000
BATCH_SLEEP=5
BUCKETS=15
### from summary_report.txt
=== Database Settings ===
name | setting
---------------------------------------+-----------
autovacuum | on
autovacuum_analyze_scale_factor | 0.1
autovacuum_analyze_threshold | 50
autovacuum_freeze_max_age | 200000000
autovacuum_max_workers | 6
autovacuum_multixact_freeze_max_age | 400000000
autovacuum_naptime | 5
autovacuum_vacuum_cost_delay | 2
autovacuum_vacuum_cost_limit | -1
autovacuum_vacuum_insert_scale_factor | 0.2
autovacuum_vacuum_insert_threshold | 1000
autovacuum_vacuum_max_threshold | 100000000
autovacuum_vacuum_scale_factor | 0.2
autovacuum_vacuum_threshold | 50
autovacuum_work_mem | -1
autovacuum_worker_slots | 16
debug_autovacuum_sort | off <<<-----------------
log_autovacuum_min_duration | 600000
max_connections | 1000
shared_buffers | 1048576
(20 rows)
=== Total Activity ===
Expanded display is on.
-[ RECORD 1 ]----------------+----------
total_n_dead_tup | 3555172
total_n_mod_since_analyze | 20714669
total_reltuples | 141795153
total_autovacuum_count | 1890
total_autoanalyze_count | 2004
total_n_tup_upd | 26995189
total_n_tup_hot_upd | 2
total_n_tup_newpage_upd | 3868918
total_n_tup_ins | 161298000
total_total_autovacuum_time | 4847051
total_total_autoanalyze_time | 695004
avg_autovacuum_time | 2564.58
avg_autoanalyze_time | 346.81
table_count | 19
### last snapshot from relstats_monitor.log
?column? | timestamp | relname | reltuples | n_dead_tup | av_count | aa_count | total_av_time | total_aa_time | n_tup_upd | n_tup_hot_upd | n_tup_ins | avg_av_time | avg_aa_time
----------+------------------------------+------------------+-----------+------------+----------+----------+---------------+---------------+-----------+---------------+-----------+-------------+-------------
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_batch_1 | 49200700 | 0 | 17 | 30 | 541848 | 46902 | 0 | 0 | 52000000 | 31873.41 | 1563.40
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_batch_2 | 44600500 | 0 | 17 | 26 | 375595 | 53137 | 0 | 0 | 49300000 | 22093.82 | 2043.73
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_batch_3 | 39800400 | 0 | 18 | 28 | 340378 | 43500 | 0 | 0 | 51200000 | 18909.89 | 1553.57
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_13_4096000 | 4094410 | 1676409 | 0 | 2 | 0 | 9029 | 1680467 | 0 | 4096000 | 0.00 | 4514.50
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_12_2048000 | 2046860 | 803638 | 2 | 4 | 931598 | 41897 | 1680863 | 0 | 2048000 | 465799.00 | 10474.25
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_11_1024000 | 1023180 | 493988 | 5 | 8 | 924292 | 54659 | 1681582 | 0 | 1024000 | 184858.40 | 6832.38
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_10_512000 | 511256 | 171819 | 12 | 19 | 728662 | 77863 | 1680512 | 0 | 512000 | 60721.83 | 4098.05
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_9_256000 | 255997 | 93717 | 24 | 41 | 251528 | 61078 | 1680115 | 0 | 256000 | 10480.33 | 1489.71
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_8_128000 | 127970 | 37869 | 42 | 60 | 166034 | 59070 | 1678672 | 0 | 128000 | 3953.19 | 984.50
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_7_64000 | 63991 | 44945 | 74 | 99 | 116048 | 54583 | 1680251 | 0 | 64000 | 1568.22 | 551.34
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_6_32000 | 32000 | 9673 | 104 | 116 | 84778 | 41956 | 1682235 | 0 | 32000 | 815.17 | 361.69
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_5_16000 | 16000 | 49469 | 186 | 190 | 74828 | 36245 | 1680406 | 0 | 16000 | 402.30 | 190.76
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_4_8000 | 7996 | 4309 | 174 | 179 | 53884 | 22075 | 1682471 | 0 | 8000 | 309.68 | 123.32
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_3_4000 | 3992 | 98236 | 214 | 215 | 49471 | 19975 | 1681034 | 0 | 4000 | 231.17 | 92.91
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_16_4000 | 3988 | 39577 | 184 | 182 | 45433 | 16430 | 1679804 | 0 | 4000 | 246.92 | 90.27
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_15_2000 | 2000 | 1753 | 199 | 194 | 42538 | 14346 | 1682234 | 0 | 2000 | 213.76 | 73.95
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_2_2000 | 1999 | 5668 | 203 | 200 | 42430 | 14802 | 1681089 | 0 | 2000 | 209.01 | 74.01
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_14_1000 | 1000 | 3934 | 203 | 199 | 38499 | 13840 | 1678710 | 1 | 1000 | 189.65 | 69.55
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_1_1000 | 903 | 55323 | 210 | 210 | 36441 | 12661 | 1681126 | 1 | 1000 | 173.53 | 60.29
(19 rows)
################################################################
## SORT ON
################################################################
### from config
BASE_DIR=$HOME/test_autovacuum_prioritization
OLTP_TABLES=16
OLTP_ROWS_START=1000
OLTP_ROWS_MULTIPLIER=2
OLTP_MAX_ROWS=5000000
BATCH_TABLES=3
BATCH_SIZE=100000
BATCH_CONNECTIONS=5
OLTP_CONNECTIONS=200
TIMEOUT=1800
OLTP_RATE=15000
BATCH_SLEEP=5
BUCKETS=15
### from summary_report.txt
=== Database Settings ===
name | setting
---------------------------------------+-----------
autovacuum | on
autovacuum_analyze_scale_factor | 0.1
autovacuum_analyze_threshold | 50
autovacuum_freeze_max_age | 200000000
autovacuum_max_workers | 6
autovacuum_multixact_freeze_max_age | 400000000
autovacuum_naptime | 5
autovacuum_vacuum_cost_delay | 2
autovacuum_vacuum_cost_limit | -1
autovacuum_vacuum_insert_scale_factor | 0.2
autovacuum_vacuum_insert_threshold | 1000
autovacuum_vacuum_max_threshold | 100000000
autovacuum_vacuum_scale_factor | 0.2
autovacuum_vacuum_threshold | 50
autovacuum_work_mem | -1
autovacuum_worker_slots | 16
debug_autovacuum_sort | on <<-----------
log_autovacuum_min_duration | 600000
max_connections | 1000
shared_buffers | 1048576
(20 rows)
=== Total Activity ===
Expanded display is on.
-[ RECORD 1 ]----------------+----------
total_n_dead_tup | 3596571
total_n_mod_since_analyze | 33295739
total_reltuples | 128920339
total_autovacuum_count | 1923
total_autoanalyze_count | 2036
total_n_tup_upd | 26992249
total_n_tup_hot_upd | 4
total_n_tup_newpage_upd | 3714297
total_n_tup_ins | 161398000
total_total_autovacuum_time | 5040892
total_total_autoanalyze_time | 735082
avg_autovacuum_time | 2621.37
avg_autoanalyze_time | 361.04
table_count | 19
### last snapshot from relstats_monitor.log
?column? | timestamp | relname | reltuples | n_dead_tup | av_count | aa_count | total_av_time | total_aa_time | n_tup_upd | n_tup_hot_upd | n_tup_ins | avg_av_time | avg_aa_time
----------+-------------------------------+------------------+-----------+------------+----------+----------+---------------+---------------+-----------+---------------+-----------+-------------+-------------
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_batch_2 | 45647800 | 0 | 19 | 32 | 397861 | 58865 | 0 | 0 | 52000000 | 20940.05 | 1839.53
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_batch_3 | 42198400 | 0 | 16 | 26 | 549624 | 48589 | 0 | 0 | 47100000 | 34351.50 | 1868.81
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_batch_1 | 32878300 | 0 | 16 | 29 | 244947 | 35940 | 0 | 0 | 53500000 | 15309.19 | 1239.31
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_13_4096000 | 4095080 | 1674368 | 0 | 2 | 0 | 11957 | 1680719 | 0 | 4096000 | 0.00 | 5978.50
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_12_2048000 | 2048010 | 785834 | 2 | 4 | 973686 | 47742 | 1683000 | 0 | 2048000 | 486843.00 | 11935.50
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_11_1024000 | 1023560 | 471569 | 5 | 8 | 1012300 | 61222 | 1677795 | 0 | 1024000 | 202460.00 | 7652.75
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_10_512000 | 513070 | 65104 | 12 | 18 | 771222 | 64852 | 1679348 | 0 | 512000 | 64268.50 | 3602.89
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_9_256000 | 255877 | 104537 | 22 | 34 | 263788 | 52513 | 1680026 | 0 | 256000 | 11990.36 | 1544.50
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_8_128000 | 127974 | 52589 | 40 | 57 | 182622 | 58427 | 1679095 | 0 | 128000 | 4565.55 | 1025.04
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_7_64000 | 63995 | 15459 | 71 | 88 | 123084 | 50789 | 1680143 | 0 | 64000 | 1733.58 | 577.15
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_6_32000 | 31998 | 18651 | 115 | 130 | 97063 | 47557 | 1678753 | 0 | 32000 | 844.03 | 365.82
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_5_16000 | 16000 | 21281 | 170 | 170 | 77925 | 35527 | 1680055 | 0 | 16000 | 458.38 | 208.98
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_4_8000 | 7997 | 23487 | 195 | 197 | 65038 | 27777 | 1682290 | 0 | 8000 | 333.53 | 141.00
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_3_4000 | 4000 | 26826 | 206 | 206 | 56811 | 22326 | 1682540 | 1 | 4000 | 275.78 | 108.38
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_16_4000 | 3999 | 25514 | 206 | 206 | 56515 | 22142 | 1681183 | 0 | 4000 | 274.34 | 107.49
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_15_2000 | 2000 | 28429 | 207 | 207 | 48227 | 19677 | 1681553 | 2 | 2000 | 232.98 | 95.06
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_2_2000 | 2000 | 28247 | 207 | 207 | 47682 | 19536 | 1679615 | 0 | 2000 | 230.35 | 94.38
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_1_1000 | 1000 | 30309 | 207 | 207 | 36442 | 16492 | 1680101 | 0 | 1000 | 176.05 | 79.67
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_14_1000 | 999 | 30314 | 207 | 207 | 36055 | 16875 | 1681990 | 1 | 1000 | 174.18 | 81.52
(19 rows)
Attachments:
[text/plain] 0001-autovacuum-score-logging-and-sort-enable-disable.txt (4.1K, ../../CAA5RZ0sw+9rEaW9taNpRZWvuLYMjRa9iibneGfB2ftNSUHT0Ww@mail.gmail.com/2-0001-autovacuum-score-logging-and-sort-enable-disable.txt)
download | inline diff:
From c24eeeb074acfb790fe44b60b2177482b9afe3c3 Mon Sep 17 00:00:00 2001
From: Ubuntu <ubuntu@ip-172-31-46-230.ec2.internal>
Date: Tue, 4 Nov 2025 15:55:40 +0000
Subject: [PATCH 1/1] autovacuum score logging and sort enable/disable
---
src/backend/postmaster/autovacuum.c | 7 ++++++-
src/backend/utils/misc/guc_parameters.dat | 10 ++++++++++
src/backend/utils/misc/guc_tables.c | 6 ++++++
src/include/postmaster/autovacuum.h | 8 ++++++++
4 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index e48bb06253b..ca9c5c615dc 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -333,6 +333,8 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
+int debug_autovacuum_sort = DEBUG_AUTOVACUUM_SORT_ON;
+
static Oid do_start_worker(void);
static void ProcessAutoVacLauncherInterrupts(void);
pg_noreturn static void AutoVacLauncherShutdown(void);
@@ -2083,6 +2085,7 @@ do_autovacuum(void)
table->oid = relid;
table->score = score;
+ elog(LOG, "adding table:%s,%lf,av=%d,aa=%d", get_rel_name(table->oid), score, dovacuum, doanalyze);
tables_to_process = lappend(tables_to_process, table);
}
@@ -2184,6 +2187,7 @@ do_autovacuum(void)
table->oid = relid;
table->score = score;
+ elog(LOG, "adding table:%s,%lf,av=1,aa=0", get_rel_name(table->oid), score);
tables_to_process = lappend(tables_to_process, table);
}
@@ -2309,7 +2313,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
- list_sort(tables_to_process, TableToProcessComparator);
+ if (debug_autovacuum_sort == DEBUG_AUTOVACUUM_SORT_ON)
+ list_sort(tables_to_process, TableToProcessComparator);
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index d6fc8333850..2bf9ce4ed27 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -3445,6 +3445,16 @@
options => 'debug_parallel_query_options',
},
+{ name => 'debug_autovacuum_sort', type => 'enum', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+ short_desc => 'Enables/Disables the autovacuum sort of eligible tables.',
+ long_desc => 'This can be useful for testing the effect of sorting eligible tables in autovacuum.',
+ flags => 'GUC_NOT_IN_SAMPLE | GUC_EXPLAIN',
+ variable => 'debug_autovacuum_sort',
+ boot_val => 'DEBUG_AUTOVACUUM_SORT_ON',
+ options => 'debug_autovacuum_sort_options',
+},
+
+
{ name => 'password_encryption', type => 'enum', context => 'PGC_USERSET', group => 'CONN_AUTH_AUTH',
short_desc => 'Chooses the algorithm for encrypting passwords.',
variable => 'Password_encryption',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 00c8376cf4d..aaa93d35187 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -404,6 +404,12 @@ static const struct config_enum_entry debug_parallel_query_options[] = {
{NULL, 0, false}
};
+static const struct config_enum_entry debug_autovacuum_sort_options[] = {
+ {"off", DEBUG_AUTOVACUUM_SORT_OFF, false},
+ {"on", DEBUG_AUTOVACUUM_SORT_ON, false},
+ {NULL, 0, false}
+};
+
static const struct config_enum_entry plan_cache_mode_options[] = {
{"auto", PLAN_CACHE_MODE_AUTO, false},
{"force_generic_plan", PLAN_CACHE_MODE_FORCE_GENERIC_PLAN, false},
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 023ac6d5fa8..32688784a06 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -25,6 +25,14 @@ typedef enum
AVW_BRINSummarizeRange,
} AutoVacuumWorkItemType;
+/* possible values for debug_autovacuum_sort */
+typedef enum
+{
+ DEBUG_AUTOVACUUM_SORT_OFF,
+ DEBUG_AUTOVACUUM_SORT_ON,
+} DebugAutovacuumSortMode;
+
+extern PGDLLIMPORT int debug_autovacuum_sort;
/* GUC variables */
extern PGDLLIMPORT bool autovacuum_start_daemon;
--
2.43.0
[text/plain] analysis.txt (12.8K, ../../CAA5RZ0sw+9rEaW9taNpRZWvuLYMjRa9iibneGfB2ftNSUHT0Ww@mail.gmail.com/3-analysis.txt)
download | inline:
################################################################
## SORT OFF
################################################################
### from config
BASE_DIR=$HOME/test_autovacuum_prioritization
OLTP_TABLES=16
OLTP_ROWS_START=1000
OLTP_ROWS_MULTIPLIER=2
OLTP_MAX_ROWS=5000000
BATCH_TABLES=3
BATCH_SIZE=100000
BATCH_CONNECTIONS=5
OLTP_CONNECTIONS=200
TIMEOUT=1800
OLTP_RATE=15000
BATCH_SLEEP=5
BUCKETS=15
### from summary_report.txt
=== Database Settings ===
name | setting
---------------------------------------+-----------
autovacuum | on
autovacuum_analyze_scale_factor | 0.1
autovacuum_analyze_threshold | 50
autovacuum_freeze_max_age | 200000000
autovacuum_max_workers | 6
autovacuum_multixact_freeze_max_age | 400000000
autovacuum_naptime | 5
autovacuum_vacuum_cost_delay | 2
autovacuum_vacuum_cost_limit | -1
autovacuum_vacuum_insert_scale_factor | 0.2
autovacuum_vacuum_insert_threshold | 1000
autovacuum_vacuum_max_threshold | 100000000
autovacuum_vacuum_scale_factor | 0.2
autovacuum_vacuum_threshold | 50
autovacuum_work_mem | -1
autovacuum_worker_slots | 16
debug_autovacuum_sort | off <<<-----------------
log_autovacuum_min_duration | 600000
max_connections | 1000
shared_buffers | 1048576
(20 rows)
=== Total Activity ===
Expanded display is on.
-[ RECORD 1 ]----------------+----------
total_n_dead_tup | 3555172
total_n_mod_since_analyze | 20714669
total_reltuples | 141795153
total_autovacuum_count | 1890
total_autoanalyze_count | 2004
total_n_tup_upd | 26995189
total_n_tup_hot_upd | 2
total_n_tup_newpage_upd | 3868918
total_n_tup_ins | 161298000
total_total_autovacuum_time | 4847051
total_total_autoanalyze_time | 695004
avg_autovacuum_time | 2564.58
avg_autoanalyze_time | 346.81
table_count | 19
### last snapshot from relstats_monitor.log
?column? | timestamp | relname | reltuples | n_dead_tup | av_count | aa_count | total_av_time | total_aa_time | n_tup_upd | n_tup_hot_upd | n_tup_ins | avg_av_time | avg_aa_time
----------+------------------------------+------------------+-----------+------------+----------+----------+---------------+---------------+-----------+---------------+-----------+-------------+-------------
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_batch_1 | 49200700 | 0 | 17 | 30 | 541848 | 46902 | 0 | 0 | 52000000 | 31873.41 | 1563.40
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_batch_2 | 44600500 | 0 | 17 | 26 | 375595 | 53137 | 0 | 0 | 49300000 | 22093.82 | 2043.73
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_batch_3 | 39800400 | 0 | 18 | 28 | 340378 | 43500 | 0 | 0 | 51200000 | 18909.89 | 1553.57
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_13_4096000 | 4094410 | 1676409 | 0 | 2 | 0 | 9029 | 1680467 | 0 | 4096000 | 0.00 | 4514.50
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_12_2048000 | 2046860 | 803638 | 2 | 4 | 931598 | 41897 | 1680863 | 0 | 2048000 | 465799.00 | 10474.25
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_11_1024000 | 1023180 | 493988 | 5 | 8 | 924292 | 54659 | 1681582 | 0 | 1024000 | 184858.40 | 6832.38
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_10_512000 | 511256 | 171819 | 12 | 19 | 728662 | 77863 | 1680512 | 0 | 512000 | 60721.83 | 4098.05
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_9_256000 | 255997 | 93717 | 24 | 41 | 251528 | 61078 | 1680115 | 0 | 256000 | 10480.33 | 1489.71
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_8_128000 | 127970 | 37869 | 42 | 60 | 166034 | 59070 | 1678672 | 0 | 128000 | 3953.19 | 984.50
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_7_64000 | 63991 | 44945 | 74 | 99 | 116048 | 54583 | 1680251 | 0 | 64000 | 1568.22 | 551.34
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_6_32000 | 32000 | 9673 | 104 | 116 | 84778 | 41956 | 1682235 | 0 | 32000 | 815.17 | 361.69
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_5_16000 | 16000 | 49469 | 186 | 190 | 74828 | 36245 | 1680406 | 0 | 16000 | 402.30 | 190.76
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_4_8000 | 7996 | 4309 | 174 | 179 | 53884 | 22075 | 1682471 | 0 | 8000 | 309.68 | 123.32
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_3_4000 | 3992 | 98236 | 214 | 215 | 49471 | 19975 | 1681034 | 0 | 4000 | 231.17 | 92.91
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_16_4000 | 3988 | 39577 | 184 | 182 | 45433 | 16430 | 1679804 | 0 | 4000 | 246.92 | 90.27
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_15_2000 | 2000 | 1753 | 199 | 194 | 42538 | 14346 | 1682234 | 0 | 2000 | 213.76 | 73.95
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_2_2000 | 1999 | 5668 | 203 | 200 | 42430 | 14802 | 1681089 | 0 | 2000 | 209.01 | 74.01
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_14_1000 | 1000 | 3934 | 203 | 199 | 38499 | 13840 | 1678710 | 1 | 1000 | 189.65 | 69.55
TAB_DATA | 2025-11-06 20:13:17.08327+00 | table_1_1000 | 903 | 55323 | 210 | 210 | 36441 | 12661 | 1681126 | 1 | 1000 | 173.53 | 60.29
(19 rows)
################################################################
## SORT ON
################################################################
### from config
BASE_DIR=$HOME/test_autovacuum_prioritization
OLTP_TABLES=16
OLTP_ROWS_START=1000
OLTP_ROWS_MULTIPLIER=2
OLTP_MAX_ROWS=5000000
BATCH_TABLES=3
BATCH_SIZE=100000
BATCH_CONNECTIONS=5
OLTP_CONNECTIONS=200
TIMEOUT=1800
OLTP_RATE=15000
BATCH_SLEEP=5
BUCKETS=15
### from summary_report.txt
=== Database Settings ===
name | setting
---------------------------------------+-----------
autovacuum | on
autovacuum_analyze_scale_factor | 0.1
autovacuum_analyze_threshold | 50
autovacuum_freeze_max_age | 200000000
autovacuum_max_workers | 6
autovacuum_multixact_freeze_max_age | 400000000
autovacuum_naptime | 5
autovacuum_vacuum_cost_delay | 2
autovacuum_vacuum_cost_limit | -1
autovacuum_vacuum_insert_scale_factor | 0.2
autovacuum_vacuum_insert_threshold | 1000
autovacuum_vacuum_max_threshold | 100000000
autovacuum_vacuum_scale_factor | 0.2
autovacuum_vacuum_threshold | 50
autovacuum_work_mem | -1
autovacuum_worker_slots | 16
debug_autovacuum_sort | on <<-----------
log_autovacuum_min_duration | 600000
max_connections | 1000
shared_buffers | 1048576
(20 rows)
=== Total Activity ===
Expanded display is on.
-[ RECORD 1 ]----------------+----------
total_n_dead_tup | 3596571
total_n_mod_since_analyze | 33295739
total_reltuples | 128920339
total_autovacuum_count | 1923
total_autoanalyze_count | 2036
total_n_tup_upd | 26992249
total_n_tup_hot_upd | 4
total_n_tup_newpage_upd | 3714297
total_n_tup_ins | 161398000
total_total_autovacuum_time | 5040892
total_total_autoanalyze_time | 735082
avg_autovacuum_time | 2621.37
avg_autoanalyze_time | 361.04
table_count | 19
### last snapshot from relstats_monitor.log
?column? | timestamp | relname | reltuples | n_dead_tup | av_count | aa_count | total_av_time | total_aa_time | n_tup_upd | n_tup_hot_upd | n_tup_ins | avg_av_time | avg_aa_time
----------+-------------------------------+------------------+-----------+------------+----------+----------+---------------+---------------+-----------+---------------+-----------+-------------+-------------
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_batch_2 | 45647800 | 0 | 19 | 32 | 397861 | 58865 | 0 | 0 | 52000000 | 20940.05 | 1839.53
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_batch_3 | 42198400 | 0 | 16 | 26 | 549624 | 48589 | 0 | 0 | 47100000 | 34351.50 | 1868.81
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_batch_1 | 32878300 | 0 | 16 | 29 | 244947 | 35940 | 0 | 0 | 53500000 | 15309.19 | 1239.31
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_13_4096000 | 4095080 | 1674368 | 0 | 2 | 0 | 11957 | 1680719 | 0 | 4096000 | 0.00 | 5978.50
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_12_2048000 | 2048010 | 785834 | 2 | 4 | 973686 | 47742 | 1683000 | 0 | 2048000 | 486843.00 | 11935.50
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_11_1024000 | 1023560 | 471569 | 5 | 8 | 1012300 | 61222 | 1677795 | 0 | 1024000 | 202460.00 | 7652.75
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_10_512000 | 513070 | 65104 | 12 | 18 | 771222 | 64852 | 1679348 | 0 | 512000 | 64268.50 | 3602.89
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_9_256000 | 255877 | 104537 | 22 | 34 | 263788 | 52513 | 1680026 | 0 | 256000 | 11990.36 | 1544.50
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_8_128000 | 127974 | 52589 | 40 | 57 | 182622 | 58427 | 1679095 | 0 | 128000 | 4565.55 | 1025.04
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_7_64000 | 63995 | 15459 | 71 | 88 | 123084 | 50789 | 1680143 | 0 | 64000 | 1733.58 | 577.15
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_6_32000 | 31998 | 18651 | 115 | 130 | 97063 | 47557 | 1678753 | 0 | 32000 | 844.03 | 365.82
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_5_16000 | 16000 | 21281 | 170 | 170 | 77925 | 35527 | 1680055 | 0 | 16000 | 458.38 | 208.98
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_4_8000 | 7997 | 23487 | 195 | 197 | 65038 | 27777 | 1682290 | 0 | 8000 | 333.53 | 141.00
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_3_4000 | 4000 | 26826 | 206 | 206 | 56811 | 22326 | 1682540 | 1 | 4000 | 275.78 | 108.38
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_16_4000 | 3999 | 25514 | 206 | 206 | 56515 | 22142 | 1681183 | 0 | 4000 | 274.34 | 107.49
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_15_2000 | 2000 | 28429 | 207 | 207 | 48227 | 19677 | 1681553 | 2 | 2000 | 232.98 | 95.06
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_2_2000 | 2000 | 28247 | 207 | 207 | 47682 | 19536 | 1679615 | 0 | 2000 | 230.35 | 94.38
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_1_1000 | 1000 | 30309 | 207 | 207 | 36442 | 16492 | 1680101 | 0 | 1000 | 176.05 | 79.67
TAB_DATA | 2025-11-06 20:46:26.073153+00 | table_14_1000 | 999 | 30314 | 207 | 207 | 36055 | 16875 | 1681990 | 1 | 1000 | 174.18 | 81.52
(19 rows)
[application/x-sh] test_autovacuum_prioritization.sh (14.9K, ../../CAA5RZ0sw+9rEaW9taNpRZWvuLYMjRa9iibneGfB2ftNSUHT0Ww@mail.gmail.com/4-test_autovacuum_prioritization.sh)
download
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-11-06 23:05 ` David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-11-06 23:05 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, 7 Nov 2025 at 11:21, Sami Imseih <samimseih@gmail.com> wrote:
> Also, I am thinking about another sorting strategy based on average
> autovacuum/autoanalyze time per table. The idea is to sort ascending by
> the greater of the two averages, so workers process quicker tables first
> instead of all workers potentially getting hung on the slowest tables.
> We can calculate the average now that v18 includes total_autovacuum_time
> and total_autoanalyze time.
>
> The way I see it, regardless of prioritization, a few large tables may
> still monopolize autovacuum workers. But at least this way, the quick tables
> get a chance to get processed first. Will this be an idea worth testing out?
This sounds like a terrible idea to me. It'll mean any table that
starts taking longer due to autovacuum neglect will have its priority
dropped for next time which will result in further neglect. If
vacuum_cost_limit is too low, then the tables in need of vacuum the
most could end up last in the queue. I also don't see how you'd handle
the fact that analyze is likely to be faster than vacuum. Tables that
only need an analyze would just come last with no regard for how
outdated the statistics are?
I'm confused at why we'd have set up our autovacuum trigger points as
they are today because we think those are good times to do a
vacuum/analyze, but then prioritise on something completely different.
Surely if we think 20% dead tuples is worth a vacuum, we must
therefore think that 40% dead tuples are even more worthwhile?! I just
cannot comprehend why we'd deviate from making the priority the
percentage over the trigger point here. If we come to the conclusion
that we want something else, then maybe our trigger point threshold
method also needs to be redefined. There certainly have been
complaints about 20% of a huge table being too much (I guess
autovacuum_vacuum_max_threshold is our answer to trying to fix that
one).
David
David
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-07 19:22 ` Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-11-07 19:22 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> On Fri, 7 Nov 2025 at 11:21, Sami Imseih <samimseih@gmail.com> wrote:
> > Also, I am thinking about another sorting strategy based on average
> > autovacuum/autoanalyze time per table. The idea is to sort ascending by
> > the greater of the two averages, so workers process quicker tables first
> > instead of all workers potentially getting hung on the slowest tables.
> > We can calculate the average now that v18 includes total_autovacuum_time
> > and total_autoanalyze time.
> >
> > The way I see it, regardless of prioritization, a few large tables may
> > still monopolize autovacuum workers. But at least this way, the quick tables
> > get a chance to get processed first. Will this be an idea worth testing out?
>
> This sounds like a terrible idea to me. It'll mean any table that
> starts taking longer due to autovacuum neglect will have its priority
> dropped for next time which will result in further neglect.
yes, that is a possibility, but I am not sure how we can actually
avoid these scenarios. The flip side is we are giving a chance
for the eligible fast tables to get more of a chance to get vacuumed,
rather than be backed because workers are all occupied on the
larger tables.
> vacuum_cost_limit is too low, then the tables in need of vacuum the
> most could end up last in the queue. I also don't see how you'd handle
> the fact that analyze is likely to be faster than vacuum. Tables that
> only need an analyze would just come last with no regard for how
> outdated the statistics are?
In the "doanalyze" case only, we will look at the average autoanalyze count,
which will push these types of tables to the front of the queue, not the last.
> I'm confused at why we'd have set up our autovacuum trigger points as
> they are today because we think those are good times to do a
> vacuum/analyze, but then prioritise on something completely different.
> Surely if we think 20% dead tuples is worth a vacuum, we must
> therefore think that 40% dead tuples are even more worthwhile?!
Sure, but thresholds alone don't indicate anything about the how quick
the table can be vacuumed, # of indexes, per table a/v settings, etc.
The average a/v time is a good proxy to determine this.
What I am suggesting here is we think beyond thresholds for
prioritization, and to give a chance for more eligible tables to get
autovacuumed rather than workers being saturated on some
of the slowest-to-vacuum tables.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-11-11 00:58 ` David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:25 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: David Rowley @ 2025-11-11 00:58 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sat, 8 Nov 2025 at 08:23, Sami Imseih <samimseih@gmail.com> wrote:
> > I'm confused at why we'd have set up our autovacuum trigger points as
> > they are today because we think those are good times to do a
> > vacuum/analyze, but then prioritise on something completely different.
> > Surely if we think 20% dead tuples is worth a vacuum, we must
> > therefore think that 40% dead tuples are even more worthwhile?!
>
> Sure, but thresholds alone don't indicate anything about the how quick
> the table can be vacuumed, # of indexes, per table a/v settings, etc.
> The average a/v time is a good proxy to determine this.
>
> What I am suggesting here is we think beyond thresholds for
> prioritization, and to give a chance for more eligible tables to get
> autovacuumed rather than workers being saturated on some
> of the slowest-to-vacuum tables.
Can you define "more eligible" here?
I think I'm not really grasping this because I don't understand why
faster-to-vacuum tables should be prioritised over slower-to-vacuum
tables. Can you explain why you think this is important?
I do understand that in your script that the OLTP tables received less
attention than unpatched, but it wasn't obvious to me why this was an
issue. If it's a case of autovacuum acting on a stale score after it
obtained the list of tables and their scores, do things look different
if we have the autovacuum worker refresh the list and scores after
it's done with a table and autovacuum_naptime has elapsed since the
list was last refreshed?
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-11 16:36 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 19:43 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
1 sibling, 2 replies; 150+ messages in thread
From: Nathan Bossart @ 2025-11-11 16:36 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Still catching up on the latest discussion, but here's a v8 patch that
amends the DEBUG3 in relation_needs_vacanalyze() to also log the score. I
might attempt to add some sort of brief documentation about autovacuum
prioritization next.
From skimming the latest discussion, I gather we might want to consider
re-sorting the list periodically. Is the idea that we'll re-sort the
remaining tables in the list, or that we'll basically restart
do_autovacuum()? If it's the latter, then we'll need to come up with some
way to decide when to stop for the current database. Right now, we just go
through pg_class and call it a day.
--
nathan
From fdb05afb9445dd00f4cad9071ef51af4f22854b0 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v8 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 211 +++++++++++++++++++++-------
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 165 insertions(+), 47 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index ed19c74bb19..326055f962f 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -310,6 +310,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +357,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1896,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1918,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2030,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2071,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2139,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2172,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2308,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2338,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2571,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2937,6 +2973,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2944,15 +2981,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2993,6 +3027,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3003,7 +3063,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3032,11 +3093,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3084,32 +3150,73 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3163,34 +3270,44 @@ relation_needs_vacanalyze(Oid relid,
* reset, because if that happens, the last vacuum and analyze counts
* will be reset too.
*/
+
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ *doanalyze = true;
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 432509277c9..3d2b5b6925a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3010,6 +3010,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v8-0001-autovacuum-scheduling-improvements.patch (13.7K, ../../aRNmEPQ18qZlLowV@nathan/2-v8-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From fdb05afb9445dd00f4cad9071ef51af4f22854b0 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v8 1/1] autovacuum scheduling improvements
---
src/backend/postmaster/autovacuum.c | 211 +++++++++++++++++++++-------
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 165 insertions(+), 47 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index ed19c74bb19..326055f962f 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -310,6 +310,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -351,7 +357,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1889,6 +1896,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1902,7 +1918,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -2014,6 +2030,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2054,11 +2071,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2114,6 +2139,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2146,11 +2172,19 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc(sizeof(TableToProcess));
+
+ table->oid = relid;
+ table->score = score;
+
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2274,6 +2308,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2302,9 +2338,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2535,7 +2571,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2937,6 +2973,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2944,15 +2981,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2993,6 +3027,32 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -3003,7 +3063,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3032,11 +3093,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3084,32 +3150,73 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3163,34 +3270,44 @@ relation_needs_vacanalyze(Oid relid,
* reset, because if that happens, the last vacuum and analyze counts
* will be reset too.
*/
+
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) vactuples / Max(vacthresh, 1));
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ *dovacuum = true;
+ *score = Max(*score, (double) instuples / Max(vacinsthresh, 1));
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ *doanalyze = true;
+ *score = Max(*score, (double) anltuples / Max(anlthresh, 1));
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 432509277c9..3d2b5b6925a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3010,6 +3010,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-11-11 19:43 ` Robert Treat <rob@xzilla.net>
2025-11-11 19:48 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Robert Treat @ 2025-11-11 19:43 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Nov 11, 2025 at 11:36 AM Nathan Bossart
<nathandbossart@gmail.com> wrote:
>
> Still catching up on the latest discussion, but here's a v8 patch that
> amends the DEBUG3 in relation_needs_vacanalyze() to also log the score. I
> might attempt to add some sort of brief documentation about autovacuum
> prioritization next.
>
> From skimming the latest discussion, I gather we might want to consider
> re-sorting the list periodically. Is the idea that we'll re-sort the
> remaining tables in the list, or that we'll basically restart
> do_autovacuum()? If it's the latter, then we'll need to come up with some
> way to decide when to stop for the current database. Right now, we just go
> through pg_class and call it a day.
>
FWIW, when I have built these types of systems in the past, and when I
wanted an aggressive recheck-type mechanism, the most common methods
involved tying it to autovacuum_max_workers. This usually was done
under the assumption that generating the list was relatively cheap and
that higher xid age would generate higher priority candidates. Of
course I also was biased towards having it be user controllable at the
database level (ie. no need to modify some control file or cron job or
whatever). To the degree those things are aligned here, there is at
least some anecdata that this is a usable setting.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 19:43 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
@ 2025-11-11 19:48 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 19:50 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-11-11 19:48 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Nov 11, 2025 at 02:43:19PM -0500, Robert Treat wrote:
> FWIW, when I have built these types of systems in the past, and when I
> wanted an aggressive recheck-type mechanism, the most common methods
> involved tying it to autovacuum_max_workers.
Would you mind elaborating on this point? Do you mean that you'd rebuild
the list every a_m_w tables, or something else?
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 19:43 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-11 19:48 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-11-11 19:50 ` Robert Treat <rob@xzilla.net>
2025-11-11 20:16 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Robert Treat @ 2025-11-11 19:50 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Nov 11, 2025 at 2:49 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
> On Tue, Nov 11, 2025 at 02:43:19PM -0500, Robert Treat wrote:
> > FWIW, when I have built these types of systems in the past, and when I
> > wanted an aggressive recheck-type mechanism, the most common methods
> > involved tying it to autovacuum_max_workers.
>
> Would you mind elaborating on this point? Do you mean that you'd rebuild
> the list every a_m_w tables, or something else?
>
Yes.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 19:43 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-11 19:48 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 19:50 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
@ 2025-11-11 20:16 ` Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 0 replies; 150+ messages in thread
From: Nathan Bossart @ 2025-11-11 20:16 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Nov 11, 2025 at 02:50:55PM -0500, Robert Treat wrote:
> On Tue, Nov 11, 2025 at 2:49 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
>> On Tue, Nov 11, 2025 at 02:43:19PM -0500, Robert Treat wrote:
>> > FWIW, when I have built these types of systems in the past, and when I
>> > wanted an aggressive recheck-type mechanism, the most common methods
>> > involved tying it to autovacuum_max_workers.
>>
>> Would you mind elaborating on this point? Do you mean that you'd rebuild
>> the list every a_m_w tables, or something else?
>
> Yes.
Interesting. With our defaults, that would mean rebuilding the list every
few tables, which seems quite aggressive. I'd start worrying about the
pg_class scanning overhead a little...
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-11-11 20:03 ` David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-11-11 20:03 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, 12 Nov 2025 at 05:36, Nathan Bossart <nathandbossart@gmail.com> wrote:
> From skimming the latest discussion, I gather we might want to consider
> re-sorting the list periodically. Is the idea that we'll re-sort the
> remaining tables in the list, or that we'll basically restart
> do_autovacuum()? If it's the latter, then we'll need to come up with some
> way to decide when to stop for the current database. Right now, we just go
> through pg_class and call it a day.
I'm still trying to work out what Sami sees in the results that he
doesn't think is good. I resuggested he try coding up the periodic
refresh-the-list code to see if it makes the thing he sees better. I
was hoping that we could get away with not doing that for stage 1 of
this. My concern there is that these change-the-way-autovacuum-works
patches seems to blow up quickly as everyone chips in with autovacuum
problems they want fixed and expect the patch to do it all.
That said, the periodic refresh probably isn't too hard. I suspected
it was something like:
/* when enough time has passed, refresh the list to ensure the
scores aren't too out-of-date */
if (time is > lastcheck + autovacuum_naptime * <something>)
{
list_free_deep(tables_to_process);
goto the_top;
}
} // end of foreach(cell, tables_to_process)
Perhaps if the test cases we're going to give this involve lengthy
autovacuum runs, then we might need that patch sooner. I'm uncertain
if that's the case with Sami's test. There were some 50GB tables, so I
imagine some of the runs could take a long time, especially so when
running standard vacuum_cost_limit.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-11 20:13 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-11-11 20:13 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, Nov 12, 2025 at 09:03:54AM +1300, David Rowley wrote:
> I'm still trying to work out what Sami sees in the results that he
> doesn't think is good. I resuggested he try coding up the periodic
> refresh-the-list code to see if it makes the thing he sees better. I
> was hoping that we could get away with not doing that for stage 1 of
> this. My concern there is that these change-the-way-autovacuum-works
> patches seems to blow up quickly as everyone chips in with autovacuum
> problems they want fixed and expect the patch to do it all.
+1
> That said, the periodic refresh probably isn't too hard. I suspected
> it was something like:
>
> /* when enough time has passed, refresh the list to ensure the
> scores aren't too out-of-date */
> if (time is > lastcheck + autovacuum_naptime * <something>)
> {
> list_free_deep(tables_to_process);
> goto the_top;
> }
> } // end of foreach(cell, tables_to_process)
My concern is that this might add already-processed tables back to the
list, so a worker might never be able to clear it. Maybe that's not a real
problem in practice for some reason, but it does feel like a step too far
for stage 1, as you said above.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-11-11 20:26 ` David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-11-11 20:26 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, 12 Nov 2025 at 09:13, Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> On Wed, Nov 12, 2025 at 09:03:54AM +1300, David Rowley wrote:
> > /* when enough time has passed, refresh the list to ensure the
> > scores aren't too out-of-date */
> > if (time is > lastcheck + autovacuum_naptime * <something>)
> > {
> > list_free_deep(tables_to_process);
> > goto the_top;
> > }
> > } // end of foreach(cell, tables_to_process)
>
> My concern is that this might add already-processed tables back to the
> list, so a worker might never be able to clear it. Maybe that's not a real
> problem in practice for some reason, but it does feel like a step too far
> for stage 1, as you said above.
Oh, that's a good point. That's a very valid concern. I guess that
could be fixed with a hashtable of vacuumed tables and skipping tables
that exist in there, but the problem with that is that the table might
genuinely need to be vacuumed again. It's a bit tricky to know when a
2nd vacuum is a legit requirement and when it's not. Figuring that out
might me more logic that this code wants to know about.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-11 23:22 ` Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Robert Treat @ 2025-11-11 23:22 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Nov 11, 2025 at 3:27 PM David Rowley <dgrowleyml@gmail.com> wrote:
> On Wed, 12 Nov 2025 at 09:13, Nathan Bossart <nathandbossart@gmail.com> wrote:
> > On Wed, Nov 12, 2025 at 09:03:54AM +1300, David Rowley wrote:
> > > /* when enough time has passed, refresh the list to ensure the
> > > scores aren't too out-of-date */
> > > if (time is > lastcheck + autovacuum_naptime * <something>)
> > > {
> > > list_free_deep(tables_to_process);
> > > goto the_top;
> > > }
> > > } // end of foreach(cell, tables_to_process)
> >
> > My concern is that this might add already-processed tables back to the
> > list, so a worker might never be able to clear it. Maybe that's not a real
> > problem in practice for some reason, but it does feel like a step too far
> > for stage 1, as you said above.
>
> Oh, that's a good point. That's a very valid concern. I guess that
> could be fixed with a hashtable of vacuumed tables and skipping tables
> that exist in there, but the problem with that is that the table might
> genuinely need to be vacuumed again. It's a bit tricky to know when a
> 2nd vacuum is a legit requirement and when it's not. Figuring that out
> might me more logic that this code wants to know about.
>
Yeah, there is a common theoretical pattern that always comes up in
these discussions where autovacuum gets stuck behind N big tables +
(AVMW - N) small tables that keep filtering up to the top of the list,
and I'm not saying that would never be a problem, but assuming the
algorithm is working correctly, this should be fairly avoidable,
because the use of xid age essentially works as a "hash of vacuumed
tables" equivalent for tracking purposes.
Walking through it, once a table is vacuumed, it should go to the
bottom of the list. The only way it crops back-up quickly is due to
significant activity on it, but even then, you need a special set of
circumstances, like it needs to be a small enough table with heavy
updates and a small autovacuum_vacuum_threshold. This type of combo
would cause the table to look like it is excessively bloated and in
need of vacuuming, but even in this scenario, eventually other tables
will get an xid age high enough that they will "out rank" the high
activity table and get their turn. TBH I'm not sure if we need to do
replanning, but in the scenarios where I have used it, having more
accurate information on the state of the database has generally been
better than relying on more stale information. Of course it isn't
100%, but the current implementation isn't either, and don't forget we
still have the failsafe_age as, well, a failsafe.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
@ 2025-11-12 20:10 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-12 22:10 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Nathan Bossart @ 2025-11-12 20:10 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Nov 11, 2025 at 06:22:36PM -0500, Robert Treat wrote:
> On Tue, Nov 11, 2025 at 3:27 PM David Rowley <dgrowleyml@gmail.com> wrote:
>> On Wed, 12 Nov 2025 at 09:13, Nathan Bossart <nathandbossart@gmail.com> wrote:
>> > My concern is that this might add already-processed tables back to the
>> > list, so a worker might never be able to clear it. Maybe that's not a real
>> > problem in practice for some reason, but it does feel like a step too far
>> > for stage 1, as you said above.
>>
>> Oh, that's a good point. That's a very valid concern. I guess that
>> could be fixed with a hashtable of vacuumed tables and skipping tables
>> that exist in there, but the problem with that is that the table might
>> genuinely need to be vacuumed again. It's a bit tricky to know when a
>> 2nd vacuum is a legit requirement and when it's not. Figuring that out
>> might me more logic that this code wants to know about.
>
> Yeah, there is a common theoretical pattern that always comes up in
> these discussions where autovacuum gets stuck behind N big tables +
> (AVMW - N) small tables that keep filtering up to the top of the list,
> and I'm not saying that would never be a problem, but assuming the
> algorithm is working correctly, this should be fairly avoidable,
> because the use of xid age essentially works as a "hash of vacuumed
> tables" equivalent for tracking purposes.
I do think re-prioritization is worth considering, but IMHO we should leave
it out of phase 1. I think it's pretty easy to reason about one round of
prioritization being okay. The order is completely arbitrary today, so how
could ordering by vacuum-related criteria make things any worse? In my
view, changing the list contents in fancier ways (e.g., adding
just-processed tables back to the list) is a step further that requires
more discussion and testing.
To be clear, I am totally for serious consideration of reprioritization,
adjusting cost delay settings, etc., but as David has repeatedly stressed,
we are unlikely to get anything committed if we try to boil the ocean. I'd
love for this thread to spin off into all kinds of other autovacuum-related
threads, but we should be taking baby steps if we want to accomplish
anything here.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-11-12 22:10 ` Sami Imseih <samimseih@gmail.com>
2025-11-12 23:51 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
1 sibling, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-11-12 22:10 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> I do think re-prioritization is worth considering, but IMHO we should leave
> it out of phase 1. I think it's pretty easy to reason about one round of
> prioritization being okay. The order is completely arbitrary today, so how
> could ordering by vacuum-related criteria make things any worse?
While it’s true that the current table order is arbitrary, that arbitrariness
naturally helps distribute vacuum work across tables of various sizes
at a given time
The proposal now is by design forcing all the top bloated table, that
will require more I/O to vacuum to be vacuumed at the same time,
by all workers. Users may observe this after they upgrade and wonder
why their I/O profile changed and perhaps slowed others non-vacuum
related processing down. They also don't have a knob to go back to
the previous behavior.
Of course, this behavior can and will happen now, but with this
prioritization, we are forcing it.
Is this a concern?
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-12 22:10 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-11-12 23:51 ` Jeremy Schneider <schneider@ardentperf.com>
2025-11-13 00:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Jeremy Schneider @ 2025-11-12 23:51 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers
> On Nov 12, 2025, at 5:10 PM, Sami Imseih <samimseih@gmail.com> wrote:
>
>
>>
>> I do think re-prioritization is worth considering, but IMHO we should leave
>> it out of phase 1. I think it's pretty easy to reason about one round of
>> prioritization being okay. The order is completely arbitrary today, so how
>> could ordering by vacuum-related criteria make things any worse?
>
> While it’s true that the current table order is arbitrary, that arbitrariness
> naturally helps distribute vacuum work across tables of various sizes
> at a given time
>
> The proposal now is by design forcing all the top bloated table, that
> will require more I/O to vacuum to be vacuumed at the same time,
> by all workers. Users may observe this after they upgrade and wonder
> why their I/O profile changed and perhaps slowed others non-vacuum
> related processing down. They also don't have a knob to go back to
> the previous behavior.
>
> Of course, this behavior can and will happen now, but with this
> prioritization, we are forcing it.
>
> Is this a concern?
It’s still possible to tune the cost delay, the number of autovacuum workers, etc - if someone needs to manage too much autovacuum I/O concurrency and dialing it back down a little bit. I think that’s sufficient
-Jeremy
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-12 22:10 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-12 23:51 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
@ 2025-11-13 00:32 ` Sami Imseih <samimseih@gmail.com>
0 siblings, 0 replies; 150+ messages in thread
From: Sami Imseih @ 2025-11-13 00:32 UTC (permalink / raw)
To: Jeremy Schneider <schneider@ardentperf.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers
>
> > On Nov 12, 2025, at 5:10 PM, Sami Imseih <samimseih@gmail.com> wrote:
> >
> >
> >>
> >> I do think re-prioritization is worth considering, but IMHO we should
> leave
> >> it out of phase 1. I think it's pretty easy to reason about one round
> of
> >> prioritization being okay. The order is completely arbitrary today, so
> how
> >> could ordering by vacuum-related criteria make things any worse?
> >
> > While it’s true that the current table order is arbitrary, that
> arbitrariness
> > naturally helps distribute vacuum work across tables of various sizes
> > at a given time
> >
> > The proposal now is by design forcing all the top bloated table, that
> > will require more I/O to vacuum to be vacuumed at the same time,
> > by all workers. Users may observe this after they upgrade and wonder
> > why their I/O profile changed and perhaps slowed others non-vacuum
> > related processing down. They also don't have a knob to go back to
> > the previous behavior.
> >
> > Of course, this behavior can and will happen now, but with this
> > prioritization, we are forcing it.
> >
> > Is this a concern?
>
> It’s still possible to tune the cost delay, the number of autovacuum
> workers, etc - if someone needs to manage too much autovacuum I/O
> concurrency and dialing it back down a little bit. I think that’s sufficient
>
Yes, the need to tune a/v for I/O( lower cost limit, higher cost delay
) will likely be
greater with this change.
--
Sami
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-11-20 14:30 ` Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Robert Haas @ 2025-11-20 14:30 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, Nov 12, 2025 at 3:10 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
> I do think re-prioritization is worth considering, but IMHO we should leave
> it out of phase 1. I think it's pretty easy to reason about one round of
> prioritization being okay. The order is completely arbitrary today, so how
> could ordering by vacuum-related criteria make things any worse? In my
> view, changing the list contents in fancier ways (e.g., adding
> just-processed tables back to the list) is a step further that requires
> more discussion and testing.
I agree with your view around reprioritization. To answer your
rhetorical question, the way that reordering the list could hurt is if
the current ordering (pg_class scan order) happened to be a
near-optimal choice. For example, suppose the last table in pg_class
order in a state where vacuuming appears to be necessary but will be
painful and/or useless (VACUUM will error, xmin will prevent all or
most tuple removal, located on an incredibly slow disk with nothing
cached, whatever). Re-sorting the list figures to move that table
earlier, which will not work out for the best. I suspect that
reprioritization actually increases the danger of this kind of failure
mode. The more aggressive you are about making sure that the
highest-priority tables actually get handled first, the more important
it is to be correct about the real order of priority.
I do think in the long term a really good system is probably going to
accumulate a bunch of extra logic to deal with cases like this. For
example, if the first table in the queue causes VACUUM to spend an
hour chugging a way and then fail with an I/O error, we would ideally
want to make sure to wait a while before retrying that table, so that
others don't get starved. But like you say, there's no need to solve
every problem at once.
What seems important to me for this patch is that we don't choose an
actively bad sort order. For instance, if we don't get the balance
between prioritizing anti-wraparound activity and controlling runaway
bloat correct, and especially if there's no way to recover by tweaking
settings, to me that's a scary scenario. I do think it's fairly
realistic for a bad choice of sort order to end up being a regression
over the current lack of a sort order. You might just be getting lucky
right now -- say, because the catalog tables all occur first in the
catalog and vacuuming those tends to be important, and among user
tables, the ones you created first are actually the ones that are most
important. That's not a particularly crazy scenario, IMHO.
Point being: I think we need to avoid the mindset that we can't be
stupider than we are now. I don't think there's any way we would
commit something that is GENERALLY stupider than we are now, but it's
not about averages. It's about whether there are specific cases that
are common enough to worry about which end up getting regressed. I'm
honestly not sure how much of a risk that is, and, again, I'm not
trying to kill the patch. It might well be that the patch is already
good enough that such scenarios will be extremely rare. However, it's
easy to get overconfident when replacing a completely unintelligent
system with a smarter one. The risk of something backfiring can
sometimes be higher than one anticipates.
One idea that might be worth considering is adding a reloption of some
kind that lets the user exert positive control over the sort order. I
know that's scope creep, so maybe it's a bad idea for that reason. But
I think it would be a better idea than Sami's proposal to score system
catalogs more highly, not so much because his idea is necessary
wrong-headed as because it doesn't help with what I see as the
principal danger here, namely, that whatever we do will sometimes turn
out to be wrong. Trying to be right 100% of the time is not going to
work out as well as having a backup plan for the cases where we are
wrong.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-11-20 16:25 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 13:07 ` Re: another autovacuum scheduling thread Dilip Kumar <dilipbalaut@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Nathan Bossart @ 2025-11-20 16:25 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Nov 20, 2025 at 09:30:42AM -0500, Robert Haas wrote:
> Point being: I think we need to avoid the mindset that we can't be
> stupider than we are now. I don't think there's any way we would
> commit something that is GENERALLY stupider than we are now, but it's
> not about averages. It's about whether there are specific cases that
> are common enough to worry about which end up getting regressed. I'm
> honestly not sure how much of a risk that is, and, again, I'm not
> trying to kill the patch. It might well be that the patch is already
> good enough that such scenarios will be extremely rare. However, it's
> easy to get overconfident when replacing a completely unintelligent
> system with a smarter one. The risk of something backfiring can
> sometimes be higher than one anticipates.
That's a fair point. The possibly-entirely-theoretical case that's in my
head is when your oldest and lowest-OID table is also the biggest and most
active. That seems like it could be a popular pattern in the field, and it
probably benefits to some degree from the sequential scan returning it
earlier. I don't know how much to worry about stuff like this.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-11-20 16:34 ` Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-11-20 16:34 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> On Thu, Nov 20, 2025 at 09:30:42AM -0500, Robert Haas wrote:
> > Point being: I think we need to avoid the mindset that we can't be
> > stupider than we are now. I don't think there's any way we would
> > commit something that is GENERALLY stupider than we are now, but it's
> > not about averages. It's about whether there are specific cases that
> > are common enough to worry about which end up getting regressed. I'm
> > honestly not sure how much of a risk that is, and, again, I'm not
> > trying to kill the patch. It might well be that the patch is already
> > good enough that such scenarios will be extremely rare. However, it's
> > easy to get overconfident when replacing a completely unintelligent
> > system with a smarter one. The risk of something backfiring can
> > sometimes be higher than one anticipates.
>
> That's a fair point. The possibly-entirely-theoretical case that's in my
> head is when your oldest and lowest-OID table is also the biggest and most
> active. That seems like it could be a popular pattern in the field, and it
> probably benefits to some degree from the sequential scan returning it
> earlier. I don't know how much to worry about stuff like this.
I think it would be difficult to introduce this new prioritization
system without a
GUC to control the prioritization behavior. Since ordering by pg_class has been
the only behavior ever since autovacuum was released, there should be a way
for users to revert back to this. The default could be the new
prioritization strategy.
Introducing new GUCs is something to be avoided if possible, but I think this
case is a clear one to me.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-11-20 18:35 ` Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Robert Haas @ 2025-11-20 18:35 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Nov 20, 2025 at 11:34 AM Sami Imseih <samimseih@gmail.com> wrote:
> I think it would be difficult to introduce this new prioritization
> system without a
> GUC to control the prioritization behavior. Since ordering by pg_class has been
> the only behavior ever since autovacuum was released, there should be a way
> for users to revert back to this. The default could be the new
> prioritization strategy.
>
> Introducing new GUCs is something to be avoided if possible, but I think this
> case is a clear one to me.
As I sort of alluded to in my previous message, I'd rather see us
introduce something that lets you get the behavior you want than
something that just lets you get back to the old behavior.
Technically, the latter is good enough to avoid any claim that we've
regressed things: you can always just the new thing off, and so by
definition there are no unavoidable regressions. But that only caters
to the scenario where the current behavior is good by accident
(because it can never be good for any other reason).
Don't take this too literally, but just mooting ideas wildly, suppose
the scoring has a wraparound component, a bloat component, and a
reloption-driven component, and the former two have a weighting factor
that can be adjusted via GUCs. If you want to shut off the new
behavior, you can setting the weighting factors to 0. If you want to
keep the new behavior but adjust the trade-off between the wraparound
and bloat components, you can adjust the relative weighting factors
between the two. If you want to take more manual control, you can use
the reloption, a choice that you can layer on top of the default
strategy or any of the alternate strategies just proposed. Of course,
making this all too complicated is a recipe for failure, but I suspect
that making it at least somewhat configurable is a good idea.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-11-20 20:21 ` Sami Imseih <samimseih@gmail.com>
1 sibling, 0 replies; 150+ messages in thread
From: Sami Imseih @ 2025-11-20 20:21 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> something that just lets you get back to the old behavior.
> Technically, the latter is good enough to avoid any claim that we've
> regressed things:
yes, that is the intention with the GUC. I am worried about
cases in which a user has no way to go back to the old
behavior if the new prioritization strategy causes pain, for
some reason.
> But that only caters
> to the scenario where the current behavior is good by accident
> (because it can never be good for any other reason).
Well, maybe it was never really good, but it was the only behavior,
and the user tuned and tested their autovacuum settings with this
behavior; whether they actually kew it's based on pg_class ordering
or not ( I know users I worked with that do not realize this ).
I think if we are to think how we can improve prioritization, the
thing in mind is what can we do so users are no longer required
to schedule manual vacuums for specific tables ( which is essentially
how users are currently prioritizing tables ). If we go to rigid strategy
that is being proposed now, will it reduce or eliminate the need for
manually scheduling? I am not so sure.
> Don't take this too literally, but just mooting ideas wildly, suppose
> the scoring has a wraparound component, a bloat component, and a
> reloption-driven component, and the former two have a weighting factor
> that can be adjusted via GUCs. If you want to shut off the new
> behavior, you can setting the weighting factors to 0.
Something like this could. be better since it can both give control over
prioritization and allows to revert to the current behavior.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-11-20 20:58 ` David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-11-20 20:58 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, 21 Nov 2025 at 07:36, Robert Haas <robertmhaas@gmail.com> wrote:
> If you want to take more manual control, you can use
> the reloption, a choice that you can layer on top of the default
> strategy or any of the alternate strategies just proposed. Of course,
> making this all too complicated is a recipe for failure, but I suspect
> that making it at least somewhat configurable is a good idea.
But it is configurable... you're free to change any of
autovacuum_freeze_max_age, autovacuum_multixact_freeze_max_age,
autovacuum_vacuum_scale_factor, autovacuum_vacuum_insert_scale_factor
and autovacuum_analyze_scale_factor, plus all the other
autovacuum_vacuum*_threshold GUCs and relptions to adjust the score.
The design is no accident. Of course, that does also affect the
eligibility for the table to be vacuumed, not just the order, but it's
not like there's no way for users to influence the order. If we really
do discover that pg_catalog tables need vacuum attention sooner, then
maybe we should consider defaulting a reloption for that, or maybe
there's only a subset of pg_catalog tables that that matters for.
For the record, I don't deny that it is possible that there is some
scenario where the pg_class order is better than sorting by the
percentage-over-threshold method, but IMO, it seems quite extreme to
go adding a series of new reloptions to weight the scores based on no
evidence that there's an actual problem or that it's even a good
solution to fixing some currently unknown problem. If we later
discover there is no issue, then reloptions are quite painful to
remove due to pg_dump (or rather failed restores). I think the vacuum
options are complex enough without risking adding a few new ones that
we don't even know are required or are even useful to anyone.
As for the GUC, I think we should at least commit the patch first and
add an open item to "Decisions to Recheck Mid-Beta" for v19 to see if
anyone still thinks a GUC is a good escape hatch, or if we'd prefer to
revert the patch because it's causing trouble. As I see it, we've got
about 6 months or maybe a bit more of testing how well this works
before we need to make a decision. My vote is to use as much of that
time as possible rather than using it to allow people to dream up
hypothetical problems that might or might not exist.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-20 21:16 ` Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Robert Haas @ 2025-11-20 21:16 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Nov 20, 2025 at 3:58 PM David Rowley <dgrowleyml@gmail.com> wrote:
> before we need to make a decision. My vote is to use as much of that
> time as possible rather than using it to allow people to dream up
> hypothetical problems that might or might not exist.
That seems a little harsh. I think the only hypothesis necessary for
my concern to be valid is the hypothesis that whatever algorithm we've
selected may not always work well. I admit that I could be wrong in
thinking so; there are plenty of heuristics in PostgreSQL that are so
effective that nobody ever cares about tuning them. But there's enough
problems with autovacuum that I don't think it's a particularly
adventurous hypothesis, either.
That said, I accept your point that even if we were to agree that
something ought to made tunable here, we would still have the problem
of deciding exactly what GUCs or reloptions to add, and that might be
hard to figure out without more information. Unfortunately, I have a
feeling that unless you or someone here is planning to make a
determined testing effort over the coming months, we're more likely to
get feedback after final release than during development or even beta.
But I do also understand that you don't want us to be paralyzed and
never move forward.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-11-20 22:12 ` David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-11-20 22:12 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, 21 Nov 2025 at 10:16, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Thu, Nov 20, 2025 at 3:58 PM David Rowley <dgrowleyml@gmail.com> wrote:
> > before we need to make a decision. My vote is to use as much of that
> > time as possible rather than using it to allow people to dream up
> > hypothetical problems that might or might not exist.
>
> That seems a little harsh.
It wasn't intended to be offensive. It's an observation that there've
been quite a few posts on this thread about various extra things we
should account for in the score without any evidence that they're
worthy of a special case. I used "dream up" since I don't recall any
of those posts arriving with evidence of an actual problem or that the
proposed solution was a valid fix for it, and that the proposed
solution didn't make something else worse.
> That said, I accept your point that even if we were to agree that
> something ought to made tunable here, we would still have the problem
> of deciding exactly what GUCs or reloptions to add, and that might be
> hard to figure out without more information. Unfortunately, I have a
> feeling that unless you or someone here is planning to make a
> determined testing effort over the coming months, we're more likely to
> get feedback after final release than during development or even beta.
You might be right. Or after a week we might discover a good reason
why the percentage-over-threshold method is rubbish and revert it. The
key is probably in the way we act from getting no negative feedback.
I suspect the most likely area the new prioritisation order could
cause issues is from the lack of randomness. Will multiple workers
working into the same database be more likely to bump into each other
somehow in a bad way? Maybe that's a good area to focus testing.
> But I do also understand that you don't want us to be paralyzed and
> never move forward.
Yeah partly, but mostly I just really doubt that this matters that
much. It's been said on this thread already that prioritisation isn't
as important as the autovacuum-configured-to-run-too-slowly issue, and
I agree with that. I just find it hard to believe that the highly
volatile pg_class order has been just perfect all these years and that
sorting by percentage-over-threshold-desc will make things worse
overall. There was mention that pg_catalog tables are first in
pg_class, but I don't really agree with that as if I create some new
tables on a fresh database, I see those getting lower ctids than any
pg_catalog table. The space for that is finite, but there's no
shortage of other reasons for user tables to become mentioned in
pg_class before catalogue tables as the database gets used. I see that
table_beginscan_catalog() uses SO_ALLOW_SYNC too, so there's an extra
layer of randomness from sync scans. I don't recall any complaints
from the order autovacuum works on tables, so, to me, it just seems
strange to think that the volatile order of pg_class just happened to
be right all these years. I suspect what's happening is that the extra
bloat or stale statistics that people get as a result of the
pg_class-order autovacuum just gets unnoticed, ignored or attended to
via adjustments to the corresponding scale_factor reloption.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-22 11:28 ` Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 20:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Robert Haas @ 2025-11-22 11:28 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Nov 20, 2025 at 5:12 PM David Rowley <dgrowleyml@gmail.com> wrote:
> It wasn't intended to be offensive.
OK.
> I suspect the most likely area the new prioritisation order could
> cause issues is from the lack of randomness. Will multiple workers
> working into the same database be more likely to bump into each other
> somehow in a bad way? Maybe that's a good area to focus testing.
I agree that lack of randomness could cause problems, but I don't see
how it could cause regressions, because the current system isn't
random, either. Even if the order of pg_class is unpredictable, it may
(depending on the workload) not change very much from one day to the
next.
> Yeah partly, but mostly I just really doubt that this matters that
> much. It's been said on this thread already that prioritisation isn't
> as important as the autovacuum-configured-to-run-too-slowly issue, and
> I agree with that. I just find it hard to believe that the highly
> volatile pg_class order has been just perfect all these years and that
> sorting by percentage-over-threshold-desc will make things worse
> overall. There was mention that pg_catalog tables are first in
> pg_class, but I don't really agree with that as if I create some new
> tables on a fresh database, I see those getting lower ctids than any
> pg_catalog table. The space for that is finite, but there's no
> shortage of other reasons for user tables to become mentioned in
> pg_class before catalogue tables as the database gets used. I see that
> table_beginscan_catalog() uses SO_ALLOW_SYNC too, so there's an extra
> layer of randomness from sync scans. I don't recall any complaints
> from the order autovacuum works on tables, so, to me, it just seems
> strange to think that the volatile order of pg_class just happened to
> be right all these years. I suspect what's happening is that the extra
> bloat or stale statistics that people get as a result of the
> pg_class-order autovacuum just gets unnoticed, ignored or attended to
> via adjustments to the corresponding scale_factor reloption.
Interesting. I don't have any real knowledge of how jumbled-up the
order of pg_class is on real production systems, and I agree that if
the answer is "it's usually quite jumbled up" then that is good news
for this patch. In any case, I'm not trying to say that prioritization
is an intrinsically bad idea, because I don't believe that. What I'm
trying to say is that there's a limited number of ways for this patch
to make things worse, and one of them is if someone is winning right
now by accident, so therefore we should think about how many people
might be in that situation. I would argue that if a large number of
users end up with a very similar pattern in terms of how pg_class is
ordered, that makes the patch higher-risk than if, as I think you're
arguing here, there's enough randomness in terms of where things end
up in pg_class to prevent any particular pattern from predominating.
In the latter case, one or two really unlucky users could end up worse
off, but that's not really an issue. What would be an issue is if we
regressed some kind of common pattern. I admit that's a bit
speculative and I'm probably being a little paranoid here: doing smart
things is typically better than doing dumb things, and what we're
doing right now is dumb.
On the other hand, once we ship something, we can't pull it back. If
it causes a problem, someone will call me at 2am and need their system
fixed right now. If my answer is "well, there are no configuration
knobs we can change and no way to get back to the old behavior and I'm
sorry you're having that problem but the only answer is for you to run
all your VACUUMs manually until two years from now when maybe the
algorithm will have been improved," it's not going to be a very good
night. After 15 years at EDB, I've learned that the problem isn't
being wrong per se; it's having no way to get out from under being
wrong. It is absolutely inevitable that I will screw up, you will
screw up, the project as a whole will screw up, and that doesn't worry
me a bit. What does worry me is the prospect that we won't have
thought hard enough about what we're going to do if and when that
happens. Most of the customers that I've gotten to work with over the
years are very gracious about things going wrong with the software as
long as there are some options to deal with the problem. I fully admit
that this patch may already be good enough that I'll never hear a
single customer complain about it, but the time to think through the
reverse scenario, where some users are unhappy, is before we ship, not
after. That necessarily involves some speculation about what might go
wrong and some of that speculation may be groundless, but speculation
causes a lot less pain than angry customers whose problems you can't
fix.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-11-22 17:28 ` Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-11-22 17:28 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> > I suspect the most likely area the new prioritisation order could
> > cause issues is from the lack of randomness. Will multiple workers
> > working into the same database be more likely to bump into each other
> > somehow in a bad way? Maybe that's a good area to focus testing.
>
> I agree that lack of randomness could cause problems, but I don't see
> how it could cause regressions, because the current system isn't
> random, either. Even if the order of pg_class is unpredictable, it may
> (depending on the workload) not change very much from one day to the
> next.
>
> > Yeah partly, but mostly I just really doubt that this matters that
> > much. It's been said on this thread already that prioritisation isn't
> > as important as the autovacuum-configured-to-run-too-slowly issue, and
> > I agree with that. I just find it hard to believe that the highly
> > volatile pg_class order has been just perfect all these years and that
> > sorting by percentage-over-threshold-desc will make things worse
> > overall. There was mention that pg_catalog tables are first in
> > pg_class, but I don't really agree with that as if I create some new
> > tables on a fresh database, I see those getting lower ctids than any
> > pg_catalog table. The space for that is finite, but there's no
> > shortage of other reasons for user tables to become mentioned in
> > pg_class before catalogue tables as the database gets used. I see that
> > table_beginscan_catalog() uses SO_ALLOW_SYNC too, so there's an extra
> > layer of randomness from sync scans. I don't recall any complaints
> > from the order autovacuum works on tables, so, to me, it just seems
> > strange to think that the volatile order of pg_class just happened to
> > be right all these years. I suspect what's happening is that the extra
> > bloat or stale statistics that people get as a result of the
> > pg_class-order autovacuum just gets unnoticed, ignored or attended to
> > via adjustments to the corresponding scale_factor reloption.
>
> Interesting. I don't have any real knowledge of how jumbled-up the
> order of pg_class is on real production systems, and I agree that if
> the answer is "it's usually quite jumbled up" then that is good news
> for this patch. In any case, I'm not trying to say that prioritization
> is an intrinsically bad idea, because I don't believe that. What I'm
> trying to say is that there's a limited number of ways for this patch
> to make things worse
What I have not been able to prove from my tests is that the processing
order of tables by autovacuum will actually make things any better or any
worse. My tests have been short 30 minute tests that count how many
vacuum cycles tables with various DML activity and sizes received.
I have not found much difference. I am also not sure how valuable
these short-duration tests are either.
On the field is where the real test occurs and it may be discovered that
the new strategy improves the majority of the cases, and there may also
be cases where the existing strategy is somehow better. Having the
ability to go back to the existing behavior seems like the best way we
can roll this out and learn over time.
These may be the only two strategies we will ever need, or we may find out
that a third strategy in which individual tables are assigned a prioritization
score will also be useful.
--
Sami
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-11-22 18:35 ` Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 15:19 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Robert Haas @ 2025-11-22 18:35 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sat, Nov 22, 2025 at 12:28 PM Sami Imseih <samimseih@gmail.com> wrote:
> What I have not been able to prove from my tests is that the processing
> order of tables by autovacuum will actually make things any better or any
> worse. My tests have been short 30 minute tests that count how many
> vacuum cycles tables with various DML activity and sizes received.
> I have not found much difference. I am also not sure how valuable
> these short-duration tests are either.
Yeah, I'm not sure that would be the right way to look for a benefit
from something like this. I think that a better test scenario might
involve figuring out how fast we can recover from a bad situation. As
we've discussed before, if VACUUM is chronically unable to keep up
with the workload, then the system is going to get into a very bad
state and there's not really any help for it. But if we start to get
into a bad situation due to some outside interference and then someone
removes the interference, we might hope that this patch would help us
get back on our feet more quickly.
For instance, suppose that we have a database with a stale replication
slot, so the oldest-XID value for the cluster keeps getting older and
older. autovacuum is probably running but it can't clean anything up.
Then at some point, the DBA realizes that bad things are happening and
drops the replication slot. You might hope that, with the patch,
autovacuum would do a better job getting the system back to a working
state. If you set up some kind of test scenario, you could ask
questions like "what is the largest age(relfrozenxid) that we observe
in the database at any point during the test?" or "from the time the
replication slot is dropped, how much time passes before
age(datfrozenxid) drops to normal?" or "what is the maximum observed
amount of bloat during the test?".
The same kind of idea could apply to anything else that stops vacuum
from running or makes it unproductive: a full table lock on a key
table, an open transaction, a table where VACUUM is failing. I
actually don't know exactly what kind of scenario would be good to
test here, because I struggle to think of a concrete scenario in which
we'd be better off with this than without it (which might be a reason
not to proceed with it, despite the fact that I think we all agree
that, from a theoretical point of view, the idea of prioritizing
sounds better than the idea of not prioritizing). But I think that if
the patch has a benefit, it won't be one where the system is in a
steady state where vacuum is able to keep up. It might be one where
we're in a steady state where vacuum is not able to keep up and things
are getting worse and worse, but the patch allows us to survive for
longer before terrible things happen. But I would say that the most
promising scenario for this patch would be something like what I
describe above, where we're not in a steady state at all: something
bad has happened and now we're trying to recover.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-11-23 09:55 ` David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-11-23 09:55 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sun, 23 Nov 2025 at 07:35, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Sat, Nov 22, 2025 at 12:28 PM Sami Imseih <samimseih@gmail.com> wrote:
> > What I have not been able to prove from my tests is that the processing
> > order of tables by autovacuum will actually make things any better or any
> > worse. My tests have been short 30 minute tests that count how many
> > vacuum cycles tables with various DML activity and sizes received.
> > I have not found much difference. I am also not sure how valuable
> > these short-duration tests are either.
>
> Yeah, I'm not sure that would be the right way to look for a benefit
> from something like this. I think that a better test scenario might
> involve figuring out how fast we can recover from a bad situation. As
> we've discussed before, if VACUUM is chronically unable to keep up
> with the workload, then the system is going to get into a very bad
> state and there's not really any help for it. But if we start to get
> into a bad situation due to some outside interference and then someone
> removes the interference, we might hope that this patch would help us
> get back on our feet more quickly.
One thing that seems to be getting forgotten again is the "/* Stop
applying cost limits from this point on */" added in 1e55e7d17 is only
going to be applied when the table *currently* being vaccumed is over
the failsafe limit. Without Nathan's patch, the worker might end up
idling along carefully obeying the cost limits on dozens of other
tables before it gets around to vacuuming the table that's over the
failsafe limit, then suddenly drop the cost delay code and rush to get
the table frozen, before Postgres stops accepting transactions. With
the patch, Nathan has added some aggressive score scaling, which
should mean any table over the failsafe limit has the highest score
and gets attended to first.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-24 19:59 ` Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Robert Haas @ 2025-11-24 19:59 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sun, Nov 23, 2025 at 4:55 AM David Rowley <dgrowleyml@gmail.com> wrote:
> One thing that seems to be getting forgotten again is the "/* Stop
> applying cost limits from this point on */" added in 1e55e7d17 is only
> going to be applied when the table *currently* being vaccumed is over
> the failsafe limit. Without Nathan's patch, the worker might end up
> idling along carefully obeying the cost limits on dozens of other
> tables before it gets around to vacuuming the table that's over the
> failsafe limit, then suddenly drop the cost delay code and rush to get
> the table frozen, before Postgres stops accepting transactions. With
> the patch, Nathan has added some aggressive score scaling, which
> should mean any table over the failsafe limit has the highest score
> and gets attended to first.
Right, so can we use that to construct a specific, concrete scenario
where we can see that the patch ends up delivering better behavior
than we have today? I think it would be a really good to have at least
one fully worked-out case where we can say "look, if you run this
series of commands without the patch, X happens, and with the patch, Y
happens, and look! Y is better."
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2026-03-05 17:03 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-05 17:03 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
My apologies for getting distracted from this work. It might be a v20 item
at this point. I haven't addressed any feedback since the v8 patch, but I
did some testing.
On Mon, Nov 24, 2025 at 02:59:33PM -0500, Robert Haas wrote:
> On Sun, Nov 23, 2025 at 4:55 AM David Rowley <dgrowleyml@gmail.com> wrote:
>> One thing that seems to be getting forgotten again is the "/* Stop
>> applying cost limits from this point on */" added in 1e55e7d17 is only
>> going to be applied when the table *currently* being vaccumed is over
>> the failsafe limit. Without Nathan's patch, the worker might end up
>> idling along carefully obeying the cost limits on dozens of other
>> tables before it gets around to vacuuming the table that's over the
>> failsafe limit, then suddenly drop the cost delay code and rush to get
>> the table frozen, before Postgres stops accepting transactions. With
>> the patch, Nathan has added some aggressive score scaling, which
>> should mean any table over the failsafe limit has the highest score
>> and gets attended to first.
>
> Right, so can we use that to construct a specific, concrete scenario
> where we can see that the patch ends up delivering better behavior
> than we have today? I think it would be a really good to have at least
> one fully worked-out case where we can say "look, if you run this
> series of commands without the patch, X happens, and with the patch, Y
> happens, and look! Y is better."
I used the xid_wraparound module to artifically induce a situation that
looked like this:
table_name | age
------------+------------
t1 | 1950000020
t2 | 1560000016
t3 | 1170000013
t4 | 780000010
t5 | 390000007
(5 rows)
Each table had 1M updates, and all other tables on the cluster were frozen.
I created the tables in reverse so that t1 is listed later in pg_class than
t5.
Without the patch, autovacuum goes straight for t5, and then processes t4,
t3, etc.:
table_name | age
------------+------------
t1 | 1950000021
t2 | 1560000017
t3 | 1170000014
t4 | 780000011
t5 | 1
(5 rows)
With the patch, it processes t1 first:
table_name | age
------------+------------
t2 | 1560000017
t3 | 1170000014
t4 | 780000011
t5 | 390000008
t1 | 1
(5 rows)
I'll admit this is a pretty extreme/contrived example, but it at least
shows the intended behavior. As alluded to elsewhere, this prioritization
work might be more useful once we are automatically adjusting the cost
limits in more cases.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-10 15:06 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-10 15:06 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Mar 05, 2026 at 11:03:50AM -0600, Nathan Bossart wrote:
> My apologies for getting distracted from this work. It might be a v20 item
> at this point. I haven't addressed any feedback since the v8 patch, but I
> did some testing.
Here's an updated patch with new GUCs that control how much each component
contributes to the autovacuum score for a table. They default to 1.0, but
can be set anywhere from 0.0 to 1.0 (inclusive). In theory, setting all of
them to 0.0 should restore the original pg_class order prioritization that
we have today. I haven't added corresponding reloptions for these GUCs, as
I'm not convinced we need them, but I can add them if folks think they
would be useful.
--
nathan
From 2a3041713338b4b5754bc69540a56f7e9adcb199 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v9 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
src/backend/postmaster/autovacuum.c | 243 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 336 insertions(+), 49 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..b5b5a63283e 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 695e187ba11..f85f77c2e1c 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -129,7 +129,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +315,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +359,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1877,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1899,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2011,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2052,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2119,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2152,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2287,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2317,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2550,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2914,6 +2952,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2921,15 +2960,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2970,6 +3006,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2980,7 +3053,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3009,11 +3083,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3061,32 +3140,76 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3135,34 +3258,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 5ee84a639d8..61d7296d1d9 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e686d88afc4..e94af29e6c3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -730,6 +730,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3250564d4ff..e65c92c6bcf 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3060,6 +3060,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v9-0001-autovacuum-scheduling-improvements.patch (24.7K, ../../abAzhNqs5F8sJDdm@nathan/2-v9-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From 2a3041713338b4b5754bc69540a56f7e9adcb199 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v9 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
src/backend/postmaster/autovacuum.c | 243 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 336 insertions(+), 49 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..b5b5a63283e 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 695e187ba11..f85f77c2e1c 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -129,7 +129,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +315,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +359,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1877,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1899,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2011,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2052,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2119,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2152,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2287,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2317,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2550,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2914,6 +2952,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2921,15 +2960,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2970,6 +3006,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2980,7 +3053,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3009,11 +3083,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3061,32 +3140,76 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3135,34 +3258,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 5ee84a639d8..61d7296d1d9 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e686d88afc4..e94af29e6c3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -730,6 +730,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3250564d4ff..e65c92c6bcf 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3060,6 +3060,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-10 16:19 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-10 16:19 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Mar 10, 2026 at 10:06:44AM -0500, Nathan Bossart wrote:
> Here's an updated patch with new GUCs that control how much each component
> contributes to the autovacuum score for a table. They default to 1.0, but
> can be set anywhere from 0.0 to 1.0 (inclusive). In theory, setting all of
> them to 0.0 should restore the original pg_class order prioritization that
> we have today. I haven't added corresponding reloptions for these GUCs, as
> I'm not convinced we need them, but I can add them if folks think they
> would be useful.
Apologies for the noise. cfbot alerted me to a missing #include.
I've been thinking about how we might eventually translate these scores
into automatic cost limit adjustments. ISTM that might be a bit difficult
because the scores are basically boundless, so we'll need to get creative.
Unfortunately, I have no concrete ideas to propose at the moment, but
that's v20 (or later) material, anyway.
--
nathan
From 99e2d5e67727f964704db63de307055beb9fe408 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v10 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
src/backend/postmaster/autovacuum.c | 244 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 337 insertions(+), 49 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..b5b5a63283e 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 695e187ba11..5f4310fa850 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -129,7 +130,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +316,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +360,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1878,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1900,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2012,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2053,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2120,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2153,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2288,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2318,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2551,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2914,6 +2953,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2921,15 +2961,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2970,6 +3007,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2980,7 +3054,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3009,11 +3084,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3061,32 +3141,76 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3135,34 +3259,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 5ee84a639d8..61d7296d1d9 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e686d88afc4..e94af29e6c3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -730,6 +730,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3250564d4ff..e65c92c6bcf 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3060,6 +3060,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v10-0001-autovacuum-scheduling-improvements.patch (24.8K, ../../abBEiOICOdbbhsEI@nathan/2-v10-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From 99e2d5e67727f964704db63de307055beb9fe408 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v10 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
src/backend/postmaster/autovacuum.c | 244 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 337 insertions(+), 49 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..b5b5a63283e 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 695e187ba11..5f4310fa850 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -129,7 +130,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +316,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +360,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1878,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1900,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2012,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2053,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2120,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2153,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2288,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2318,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2551,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2914,6 +2953,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2921,15 +2961,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2970,6 +3007,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2980,7 +3054,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3009,11 +3084,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3061,32 +3141,76 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ Oid xid_age;
+ Oid mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3135,34 +3259,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 5ee84a639d8..61d7296d1d9 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e686d88afc4..e94af29e6c3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -730,6 +730,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3250564d4ff..e65c92c6bcf 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3060,6 +3060,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-11 00:11 ` Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2026-03-11 00:11 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> Here's an updated patch with new GUCs that control how much each component
> contributes to the autovacuum score for a table. They default to 1.0, but
> can be set anywhere from 0.0 to 1.0 (inclusive). In theory, setting all of
> them to 0.0 should restore the original pg_class order prioritization that
> we have today. I haven't added corresponding reloptions for these GUCs, as
> I'm not convinced we need them, but I can add them if folks think they
> would be useful.
Starting with GUCs is OK by me.
Just a few things:
1/
+ Oid xid_age;
+ Oid mxid_age;
Is using Oid here intentional? I'm curious why not use uint32 for clarity?
2/
The new GUC docs says "...component of the score...", but without
introducing the concept of the prioritization score.
I think we should expand a bit more on this topic to help a user
understand and tune these more effectively. Attached is my
proposal for the docs. I tried to keep it informative without
being too verbose, and avoided making specific recommendations.
--
Sami Imseih
Amazon Web Services (AWS)
Attachments:
[application/octet-stream] v1-0001-autovacuum-scheduling-improvements-docs.patch (4.5K, ../../CAA5RZ0seLxDmH9UftUA3vS8DrW+7T9HV5XK8yAuQxA8BaRkg2g@mail.gmail.com/2-v1-0001-autovacuum-scheduling-improvements-docs.patch)
download | inline diff:
From eee71cdfdaff3295d52c1213d47ec1754e87a1f8 Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Tue, 10 Mar 2026 19:04:03 -0500
Subject: [PATCH v1 1/1] autovacuum scheduling improvements - docs
---
doc/src/sgml/maintenance.sgml | 95 +++++++++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 7c958b06273..16b50f8e5b6 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1054,6 +1054,99 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
not automatically interrupted.
</para>
+ <sect3 id="autovacuum-priority">
+ <title>Processing Priority</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it picks a
+ database, then it orders the tables within that database.
+ </para>
+
+ <para>
+ The launcher first checks for databases at risk of wraparound,
+ with transaction ID wraparound taking precedence over multixact
+ wraparound. If no database is at risk, the least recently
+ auto-vacuumed database is selected. Databases that have never been
+ connected to, or that have had no activity since the statistics were
+ last reset, are not considered except when at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker builds a list of all tables
+ that need vacuuming or analyzing and sorts them by a
+ <firstterm>priority score</firstterm>. Tables with higher scores are
+ processed first.
+ </para>
+
+ <para>
+ The score for each table is calculated as the maximum of several
+ component scores, each representing how far the table has exceeded a
+ particular threshold. Each component is multiplied by a configurable
+ weight parameter:
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>: the ratio of
+ dead tuples to the table's vacuum threshold. For example, if a table
+ has 100 dead tuples and its vacuum threshold is 80, this component's
+ score is 1.25.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>: the ratio
+ of inserted tuples (since the last vacuum) to the table's insert
+ vacuum threshold.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>: the ratio of
+ modified tuples (since the last analyze) to the table's analyze
+ threshold.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>: the ratio of
+ the table's transaction ID age
+ (<structfield>relfrozenxid</structfield>) to
+ <xref linkend="guc-autovacuum-freeze-max-age"/>. This component is
+ only considered for tables that have already exceeded their freeze max
+ age.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>: the
+ ratio of the table's multixact age
+ (<structfield>relminmxid</structfield>) to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Like the
+ freeze score, this is only considered for tables past their multixact
+ freeze max age.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ The final score is the maximum of these weighted components.
+ </para>
+
+ <para>
+ Tables that are approaching transaction ID or multixact wraparound receive
+ additional priority. Once a table's age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/> or
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>, its freeze score is
+ scaled aggressively so that it sorts toward the top of the list.
+ </para>
+
+ <para>
+ All weights default to 1.0. Reducing a weight to a value below 1.0
+ decreases the influence of that component on the final score, making
+ tables that exceed that particular threshold less likely to be processed
+ first.
+ </para>
+
<warning>
<para>
Regularly running commands that acquire locks conflicting with a
@@ -1061,6 +1154,8 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ </sect3>
</sect2>
</sect1>
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2026-03-11 01:08 ` Sami Imseih <samimseih@gmail.com>
2026-03-11 03:56 ` Re: another autovacuum scheduling thread wenhui qiu <qiuwenhuifx@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Sami Imseih @ 2026-03-11 01:08 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> Just a few things:
>
> 1/
> + Oid xid_age;
> + Oid mxid_age;
>
> Is using Oid here intentional? I'm curious why not use uint32 for clarity?
>
> 2/
> The new GUC docs says "...component of the score...", but without
> introducing the concept of the prioritization score.
> I think we should expand a bit more on this topic to help a user
> understand and tune these more effectively. Attached is my
> proposal for the docs. I tried to keep it informative without
> being too verbose, and avoided making specific recommendations.
My apologies. I found something else that may need
addressing.
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double)
xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double)
mxid_age / 100000000));
+
The current scaling calculation for force_vacuum could lead to
exorbitantly high scores.
Using DEBUG3 and consume_xids_until(2000000000), notice how the score goes
from 7.93 to 661828682916018.125 once past failsafe age.
36), anl: 0 (threshold 97929), score: 7.930
2026-03-10 19:41:11.979 CDT [74007] DEBUG: foo: vac: 0 (threshold
195809), ins: 0 (threshold 176836), anl: 0 (threshold 97929), score:
7.930
2026-03-10 19:41:32.062 CDT [74038] DEBUG: foo: vac: 0 (threshold
195809), ins: 0 (threshold 176836), anl: 0 (threshold 97929), score:
661828682916018.125
2026-03-10 19:41:32.063 CDT [74038] DEBUG: foo: vac: 0 (threshold
195809), ins: 0 (threshold 176836), anl: 0 (threshold 97929), score:
661828682916018.125
2026-03-10 19:41:51.961 CDT [74066] DEBUG: foo: vac: 0 (threshold
195809), ins: 0 (threshold 176836), anl: 0 (threshold 97929), score:
26761249940789168.000
Do you think it will be better to just to add the age to the
score?
if (xid_age >= effective_xid_failsafe_age)
xid_score += (double) xid_age;
if (mxid_age >= effective_mxid_failsafe_age)
mxid_score += (double) mxid_age
For most cases, this should be high enough to to make the
score high enough to sort to the top, as mentioned in the
comments:
+ * As in vacuum_xid_failsafe_check(), the effective
failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could
result in a low
+ * score even if the table surpasses the failsafe
settings. However,
+ * this is a strange enough corner case that we don't
bother trying to
+ * handle it.
+ */
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2026-03-11 03:56 ` wenhui qiu <qiuwenhuifx@gmail.com>
1 sibling, 0 replies; 150+ messages in thread
From: wenhui qiu @ 2026-03-11 03:56 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
HI Sami
> Do you think it will be better to just to add the age to the
> score?
>
> if (xid_age >= effective_xid_failsafe_age)
> xid_score += (double) xid_age;
> if (mxid_age >= effective_mxid_failsafe_age)
> mxid_score += (double) mxid_age
>
> For most cases, this should be high enough to to make the
> score high enough to sort to the top, as mentioned in the
> comments:
Agree, +1.The calculation resulted in a very large number. I previously
suggested modifying the algorithm like this:
effective_xid_failsafe_age = (vacuum_failsafe_age +
autovacuum_freeze_max_age) / 2.0. Typically, the `vacuum_failsafe_age`
parameter is rarely adjusted by DBAs.My view has always been that tables
whose age cannot be reduced should be prioritized, while we should try to
avoid tables whose age is already close to vacuum_failsafe_age.
Thanks
On Wed, Mar 11, 2026 at 9:08 AM Sami Imseih <samimseih@gmail.com> wrote:
> > Just a few things:
> >
> > 1/
> > + Oid xid_age;
> > + Oid mxid_age;
> >
> > Is using Oid here intentional? I'm curious why not use uint32 for
> clarity?
> >
> > 2/
> > The new GUC docs says "...component of the score...", but without
> > introducing the concept of the prioritization score.
> > I think we should expand a bit more on this topic to help a user
> > understand and tune these more effectively. Attached is my
> > proposal for the docs. I tried to keep it informative without
> > being too verbose, and avoided making specific recommendations.
>
> My apologies. I found something else that may need
> addressing.
>
> + if (xid_age >= effective_xid_failsafe_age)
> + xid_score = pow(xid_score, Max(1.0, (double)
> xid_age / 100000000));
> + if (mxid_age >= effective_mxid_failsafe_age)
> + mxid_score = pow(mxid_score, Max(1.0, (double)
> mxid_age / 100000000));
> +
>
> The current scaling calculation for force_vacuum could lead to
> exorbitantly high scores.
> Using DEBUG3 and consume_xids_until(2000000000), notice how the score goes
> from 7.93 to 661828682916018.125 once past failsafe age.
>
> 36), anl: 0 (threshold 97929), score: 7.930
> 2026-03-10 19:41:11.979 CDT [74007] DEBUG: foo: vac: 0 (threshold
> 195809), ins: 0 (threshold 176836), anl: 0 (threshold 97929), score:
> 7.930
> 2026-03-10 19:41:32.062 CDT [74038] DEBUG: foo: vac: 0 (threshold
> 195809), ins: 0 (threshold 176836), anl: 0 (threshold 97929), score:
> 661828682916018.125
> 2026-03-10 19:41:32.063 CDT [74038] DEBUG: foo: vac: 0 (threshold
> 195809), ins: 0 (threshold 176836), anl: 0 (threshold 97929), score:
> 661828682916018.125
> 2026-03-10 19:41:51.961 CDT [74066] DEBUG: foo: vac: 0 (threshold
> 195809), ins: 0 (threshold 176836), anl: 0 (threshold 97929), score:
> 26761249940789168.000
>
> Do you think it will be better to just to add the age to the
> score?
>
> if (xid_age >= effective_xid_failsafe_age)
> xid_score += (double) xid_age;
> if (mxid_age >= effective_mxid_failsafe_age)
> mxid_score += (double) mxid_age
>
> For most cases, this should be high enough to to make the
> score high enough to sort to the top, as mentioned in the
> comments:
>
> + * As in vacuum_xid_failsafe_check(), the effective
> failsafe age is no
> + * less than 105% the value of the respective
> *_freeze_max_age
> + * parameter. Note that per-table settings could
> result in a low
> + * score even if the table surpasses the failsafe
> settings. However,
> + * this is a strange enough corner case that we don't
> bother trying to
> + * handle it.
> + */
>
> --
> Sami Imseih
> Amazon Web Services (AWS)
>
>
>
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2026-03-11 15:53 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-11 15:53 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Mar 10, 2026 at 08:08:26PM -0500, Sami Imseih wrote:
>> + Oid xid_age;
>> + Oid mxid_age;
>>
>> Is using Oid here intentional? I'm curious why not use uint32 for clarity?
Fixed.
>> The new GUC docs says "...component of the score...", but without
>> introducing the concept of the prioritization score.
>> I think we should expand a bit more on this topic to help a user
>> understand and tune these more effectively. Attached is my
>> proposal for the docs. I tried to keep it informative without
>> being too verbose, and avoided making specific recommendations.
Good idea. I put my own spin on it in the attached. Please let me know
what you think.
> The current scaling calculation for force_vacuum could lead to
> exorbitantly high scores.
> Using DEBUG3 and consume_xids_until(2000000000), notice how the score goes
> from 7.93 to 661828682916018.125 once past failsafe age.
>
> [...]
>
> Do you think it will be better to just to add the age to the
> score?
I mean, that's kind of the point. Once a table surpasses one of the
failsafe thresholds, we want its score to be so exorbitantly high that
autovacuum is all but guaranteed to process it first. I see no particular
advantage to tempering the score in that case.
--
nathan
From 2f0826f85ded08ee32a9ac0f5b0bd55d4b96d842 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v11 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
doc/src/sgml/maintenance.sgml | 92 +++++++
src/backend/postmaster/autovacuum.c | 244 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 429 insertions(+), 49 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..b5b5a63283e 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 7c958b06273..b609f05be07 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1061,6 +1061,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 695e187ba11..4160de3d25b 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -129,7 +130,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +316,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +360,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1878,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1900,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2012,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2053,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2120,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2153,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2288,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2318,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2551,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2914,6 +2953,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2921,15 +2961,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2970,6 +3007,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2980,7 +3054,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3009,11 +3084,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3061,32 +3141,76 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3135,34 +3259,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index a5a0edf2534..73bbd44c565 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e686d88afc4..e94af29e6c3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -730,6 +730,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3da19d41413..dccb28fd8e1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3063,6 +3063,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v11-0001-autovacuum-scheduling-improvements.patch (29.2K, ../../abGP87A3JPIXDG2I@nathan/2-v11-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From 2f0826f85ded08ee32a9ac0f5b0bd55d4b96d842 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v11 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
doc/src/sgml/maintenance.sgml | 92 +++++++
src/backend/postmaster/autovacuum.c | 244 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 429 insertions(+), 49 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..b5b5a63283e 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 7c958b06273..b609f05be07 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1061,6 +1061,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 695e187ba11..4160de3d25b 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -129,7 +130,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +316,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +360,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1878,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1900,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2012,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2053,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2120,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2153,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2288,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2318,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2551,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2914,6 +2953,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2921,15 +2961,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2970,6 +3007,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2980,7 +3054,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3009,11 +3084,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ TransactionId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ /* initialize variables that aren't guaranteed to be set below */
+ *score = 0.0;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3061,32 +3141,76 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
force_vacuum = MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(relminmxid, multiForceLimit);
}
- *wraparound = force_vacuum;
+ *wraparound = *dovacuum = force_vacuum;
+
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ }
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3135,34 +3259,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index a5a0edf2534..73bbd44c565 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e686d88afc4..e94af29e6c3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -730,6 +730,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3da19d41413..dccb28fd8e1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3063,6 +3063,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-11 17:08 ` Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2026-03-11 17:08 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> Good idea. I put my own spin on it in the attached. Please let me know
> what you think.
This look OK to me.
> > The current scaling calculation for force_vacuum could lead to
> > exorbitantly high scores.
> > Using DEBUG3 and consume_xids_until(2000000000), notice how the score goes
> > from 7.93 to 661828682916018.125 once past failsafe age.
> >
> > [...]
> >
> > Do you think it will be better to just to add the age to the
> > score?
>
> I mean, that's kind of the point. Once a table surpasses one of the
> failsafe thresholds, we want its score to be so exorbitantly high that
> autovacuum is all but guaranteed to process it first. I see no particular
> advantage to tempering the score in that case.
The main issue is that the scores can reach quadrillions, or even billions,
which feels excessive, especially if exposed in DEBUG3 or in a future
prioritization view.
So scaling the scores down seems like the right thing to do.
We could also do this by dividing the score by a large constant, or
use log10 to compress
the score. Both methods should keep the sort order unchanged.
However, If everyone agrees with the current approach, I will not push
back further.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2026-03-11 17:28 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-11 17:28 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, Mar 11, 2026 at 12:08:52PM -0500, Sami Imseih wrote:
> The main issue is that the scores can reach quadrillions, or even billions,
> which feels excessive, especially if exposed in DEBUG3 or in a future
> prioritization view.
But why is that an issue? Because the number looks big when there's
extremely verbose logging enabled? I'm not following your objection. IMHO
we _want_ the score to be excessively high in these cases so that there's
basically zero chance a table with unreasonable bloat takes priority. This
was discussed a bit upthread [0].
[0] https://postgr.es/m/CAApHDvqrd%3DSHVUytdRj55OWnLH98Rvtzqam5zq2f4XKRZa7t9Q%40mail.gmail.com
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-11 17:59 ` Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2026-03-11 17:59 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
>
> On Wed, Mar 11, 2026 at 12:08:52PM -0500, Sami Imseih wrote:
> > The main issue is that the scores can reach quadrillions, or even
> billions,
> > which feels excessive, especially if exposed in DEBUG3 or in a future
> > prioritization view.
>
> But why is that an issue? Because the number looks big when there's
> extremely verbose logging enabled? I'm not following your objection.
Yes, purely the looks of such an excessively large number could look wrong
to a user.
Putting on my user hat, I would be confused and honestly think this is a
bug in the
calculation. If we weren’t exposing the numbers, I would not care.
But, this could just be me.
This comment "this component increases greatly once the age surpasses" is
perhaps
good enough.
we _want_ the score to be excessively high in these cases so that there's
> basically zero chance a table with unreasonable bloat takes priority. This
> was discussed a bit upthread [0].
>
> [0]
> https://postgr.es/m/CAApHDvqrd%3DSHVUytdRj55OWnLH98Rvtzqam5zq2f4XKRZa7t9Q%40mail.gmail.com
>
Yes, I definitely agree with this.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2026-03-12 19:20 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-12 19:20 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
I'm debating whether I should move forward with committing this [0] for
v19. On one hand, I think I've addressed all the latest feedback, and I'm
not aware of any sustained objections to the approach. But on the other
hand, there hasn't been much discussion since November (my fault), and I
can't quite tell if this patch has enough support. At the moment, I'm
leaning towards committing it, but I'm curious what folks think.
[0] https://postgr.es/m/attachment/191721/v11-0001-autovacuum-scheduling-improvements.patch
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-17 23:06 ` David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2026-03-17 23:06 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, 13 Mar 2026 at 08:20, Nathan Bossart <nathandbossart@gmail.com> wrote:
> I'm debating whether I should move forward with committing this [0] for
> v19. On one hand, I think I've addressed all the latest feedback, and I'm
> not aware of any sustained objections to the approach. But on the other
> hand, there hasn't been much discussion since November (my fault), and I
> can't quite tell if this patch has enough support. At the moment, I'm
> leaning towards committing it, but I'm curious what folks think.
I think it would have been better to have done this about 3 months
ago, but I think it's probably still fine to do now. Feature freeze is
still quite a long way from release. I do expect that the most likely
time that someone would find issues with this change would be during
beta or RC, as that's when people would give PostgreSQL production
workloads to try out. During the dev cycle, I expect it's *mostly*
just hackers giving the database toy workloads in a very targeted way
to something specific that they're hacking on. Anyway, now since
you've added the GUCs to control the weights, there's a way for users
to have some influence, so at least there's an escape hatch.
I think the GUCs are probably a good idea. I expect the most likely
change that people might want to make would be to raise the priority
of analyze over vacuum since that's often much faster to complete. We
know that some workloads are very sensitive to outdated statistics.
On the other hand, we shouldn't be taking adding 5 new autovacuum GUCs
lightly as there are already so many. If we are going to come up with
something better than this in a few years then it could be better to
wait to reduce the pain of having to remove GUCs in the future. I
don't personally have any better ideas, so I'd rather see it go in
than not.
I didn't look at the patch in detail, but noticed that you might want
to add a "See Section N.N.N for more information." to the new GUCs to
link to the text you've added on how they're used.
Do you think it's worth making the call to
list_sort(tables_to_process, TableToProcessComparator) conditional on
a variable set like: sort_required |= (score != 0.0);? I recall that
someone had concerns that the actual sort itself could add overhead.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2026-03-18 16:09 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-18 16:09 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, Mar 18, 2026 at 12:06:34PM +1300, David Rowley wrote:
> I think it would have been better to have done this about 3 months
> ago, but I think it's probably still fine to do now. Feature freeze is
> still quite a long way from release. I do expect that the most likely
> time that someone would find issues with this change would be during
> beta or RC, as that's when people would give PostgreSQL production
> workloads to try out. During the dev cycle, I expect it's *mostly*
> just hackers giving the database toy workloads in a very targeted way
> to something specific that they're hacking on. Anyway, now since
> you've added the GUCs to control the weights, there's a way for users
> to have some influence, so at least there's an escape hatch.
Thanks for chiming in.
> I think the GUCs are probably a good idea. I expect the most likely
> change that people might want to make would be to raise the priority
> of analyze over vacuum since that's often much faster to complete. We
> know that some workloads are very sensitive to outdated statistics.
>
> On the other hand, we shouldn't be taking adding 5 new autovacuum GUCs
> lightly as there are already so many. If we are going to come up with
> something better than this in a few years then it could be better to
> wait to reduce the pain of having to remove GUCs in the future. I
> don't personally have any better ideas, so I'd rather see it go in
> than not.
Yeah, adding these GUCs feels a bit like etching in stone, but if folks
want configurability, and nobody has better ideas, I'm not sure what else
to do.
> I didn't look at the patch in detail, but noticed that you might want
> to add a "See Section N.N.N for more information." to the new GUCs to
> link to the text you've added on how they're used.
Good idea. I've added that.
> Do you think it's worth making the call to
> list_sort(tables_to_process, TableToProcessComparator) conditional on
> a variable set like: sort_required |= (score != 0.0);? I recall that
> someone had concerns that the actual sort itself could add overhead.
I don't think it's necessary. I tested sorting 1M and 10M tables with
randomly generated scores (on my MacBook, with assertions enabled). The
former took ~150 milliseconds, and the latter took ~1770 milliseconds. I
suspect there are far bigger problems to worry about if you have anywhere
near that many tables.
--
nathan
From f248edb07e1a8cd5293b22754c5402e53abb027f Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v12 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
doc/src/sgml/maintenance.sgml | 92 +++++++
src/backend/postmaster/autovacuum.c | 243 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 429 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 7c958b06273..b609f05be07 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1061,6 +1061,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 219673db930..7931a7cfcaf 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -312,6 +317,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +361,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1867,6 +1879,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1880,7 +1901,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1992,6 +2013,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2032,11 +2054,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2092,6 +2121,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2124,11 +2154,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2252,6 +2289,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2280,9 +2319,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2513,7 +2552,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2915,6 +2954,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2922,15 +2962,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2971,6 +3008,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2981,7 +3055,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3010,11 +3085,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ *score = 0.0;
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3062,17 +3142,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3081,13 +3161,58 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3136,34 +3261,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..b47bc534ee1 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 174e2798443..bc4e6444a2d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3088,6 +3088,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v12-0001-autovacuum-scheduling-improvements.patch (29.4K, ../../abrOO8GCgkhMXvcq@nathan/2-v12-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From f248edb07e1a8cd5293b22754c5402e53abb027f Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v12 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
doc/src/sgml/maintenance.sgml | 92 +++++++
src/backend/postmaster/autovacuum.c | 243 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 429 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 7c958b06273..b609f05be07 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1061,6 +1061,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 219673db930..7931a7cfcaf 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -312,6 +317,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +361,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1867,6 +1879,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1880,7 +1901,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1992,6 +2013,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2032,11 +2054,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2092,6 +2121,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2124,11 +2154,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2252,6 +2289,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2280,9 +2319,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2513,7 +2552,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2915,6 +2954,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2922,15 +2962,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2971,6 +3008,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2981,7 +3055,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3010,11 +3085,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ *score = 0.0;
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3062,17 +3142,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3081,13 +3161,58 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3136,34 +3261,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..b47bc534ee1 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 174e2798443..bc4e6444a2d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3088,6 +3088,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-19 09:55 ` Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Greg Burd @ 2026-03-19 09:55 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; David Rowley <dgrowleyml@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, Mar 18, 2026, at 12:09 PM, Nathan Bossart wrote:
> On Wed, Mar 18, 2026 at 12:06:34PM +1300, David Rowley wrote:
>> I think it would have been better to have done this about 3 months
>> ago, but I think it's probably still fine to do now. Feature freeze is
>> still quite a long way from release. I do expect that the most likely
>> time that someone would find issues with this change would be during
>> beta or RC, as that's when people would give PostgreSQL production
>> workloads to try out. During the dev cycle, I expect it's *mostly*
>> just hackers giving the database toy workloads in a very targeted way
>> to something specific that they're hacking on. Anyway, now since
>> you've added the GUCs to control the weights, there's a way for users
>> to have some influence, so at least there's an escape hatch.
>
> Thanks for chiming in.
Hey Nathan, et. al., I'll chime in too! Apologies in advance for the length of the message. I need to learn how to be more concise...
First off, thanks for looking into this! I think your work is a significant improvement over where we are today and should be in v19. Put simply, I think that the v12 patch is no worse than pg_class order we use today and in many cases much better. That said, I think we should tweak it a bit more while we're at it.
>> I think the GUCs are probably a good idea. I expect the most likely
>> change that people might want to make would be to raise the priority
>> of analyze over vacuum since that's often much faster to complete. We
>> know that some workloads are very sensitive to outdated statistics.
>>
>> On the other hand, we shouldn't be taking adding 5 new autovacuum GUCs
>> lightly as there are already so many. If we are going to come up with
>> something better than this in a few years then it could be better to
>> wait to reduce the pain of having to remove GUCs in the future. I
>> don't personally have any better ideas, so I'd rather see it go in
>> than not.
>
> Yeah, adding these GUCs feels a bit like etching in stone, but if folks
> want configurability, and nobody has better ideas, I'm not sure what else
> to do.
I'm late in the review process. I know David Rowley proposed the unified scoring approach that became the foundation of this patch, and I think that's a great direction. However, I'm concerned that the patch's default scoring weights don't give XID-age urgency sufficient priority over dead-tuple urgency. The weight GUCs (autovacuum_vacuum_score_weight, etc.) can address this, but they max at 1.0, meaning you can only reduce dead-tuple priority, not increase XID priority.
>> I didn't look at the patch in detail, but noticed that you might want
>> to add a "See Section N.N.N for more information." to the new GUCs to
>> link to the text you've added on how they're used.
>
> Good idea. I've added that.
>
>> Do you think it's worth making the call to
>> list_sort(tables_to_process, TableToProcessComparator) conditional on
>> a variable set like: sort_required |= (score != 0.0);? I recall that
>> someone had concerns that the actual sort itself could add overhead.
>
> I don't think it's necessary. I tested sorting 1M and 10M tables with
> randomly generated scores (on my MacBook, with assertions enabled). The
> former took ~150 milliseconds, and the latter took ~1770 milliseconds. I
> suspect there are far bigger problems to worry about if you have anywhere
> near that many tables.
>
> --
> nathan
>
> Attachments:
> * v12-0001-autovacuum-scheduling-improvements.patch
I decided to model before/after behavior using discrete-event simulation. With a bit of LLM help, we no have autovacuum_simulation.py and autovacuum_simulation_fixed.py to compare all three approaches (Before/v12/Proposed small fix to v12) across 20 runs with randomized OID assignments.
Results from the models are at the end of this email. They simulate the before/after/fixed behavior of the v12 autovacuum scheduling patch and a suggested fix (I'll get to why in a minute).
They test a contention scenario: 100 tables (5 already past freeze_max_age, 15 approaching it at staggered rates, and 80 high-churn tables constantly exceeding their dead-tuple vacuum thresholds) competing for 3 autovacuum workers over 7 days. I'm sure there are other scenarios that we could test, this felt representational enough to me based on what I've seen.
With the first test, the "Before" scheduler processes tables in OID order; the "After" scheduler uses the patch's urgency scoring with all default GUC values.
The model of the v12 patch produces a clear system-wide improvement: the number of tables simultaneously past freeze_max_age peaks at roughly 20 under score-based scheduling versus 80+ under OID order. This happens because high-churn tables earn high dead-tuple scores, get vacuumed frequently, and their relfrozenxid ages reset as a side effect - preventing them from ever reaching freeze_max_age. Under OID order, high-OID active tables are *starved* (for no other reason than they were created later than their counterparts!) and their XID ages grow needlessly into the danger zone. The v12 patch also correctly prioritizes the most dangerous force-vacuum tables first (those closest to failsafe age), whereas OID order's success with any particular table is coincidental.
But there's a downside in v12, the simulation reveals a scoring-scale concern under default weights. Active tables accumulate dead-tuple scores of 18–70+ within minutes of their last vacuum, while force-vacuum tables that have just crossed freeze_max_age carry XID scores of only 1.0–4.0 (age/freeze_max_age). The exponential boost doesn't activate until failsafe age (1.6B), which is 8× the freeze_max_age threshold. In the gap between 200M and 1.6B, force-vacuum tables are consistently outranked by ordinary dead-tuple work. In the tested scenario this meant only 2 of 20 at-risk tables were actually vacuumed by the score-based scheduler (versus 5 by OID due to luck), and average per-table wraparound exposure was 26% worse.
One possible remedy within the current design would be to either raise the default autovacuum_freeze_score_weight or apply a floor multiplier when force_vacuum is true. For example, ensuring any force-vacuum score is at least as large as the maximum non-force-vacuum score in the current candidate set. Alternatively, the weight GUC range could be expanded above 1.0 to allow DBAs to explicitly boost XID-age priority. The existing 0.0–1.0 range only allows reducing component priority, which makes it difficult to express "wraparound prevention is more important than bloat control.
Tiered sorting using the existing wraparound flag might be the simplest and safest fix. While the thread discussed using scoring with exponential boost to prioritize wraparound tables, I think a two-tier approach (wraparound tier vs routine tier) would be more robust. We already treat wraparound as non-negotiable (force_vacuum bypasses av_enabled, triggers emergency behavior). The scoring system should reflect this by making wraparound a hard priority tier, not a score component competing with bloat cleanup.
Instead of sorting solely by score, sort first by wraparound, then by score within each tier. The score computation stays exactly as-is; it's only used for relative ordering among force_vacuum tables and among non-force-vacuum tables.
Something like:
/*
* Comparison function for sorting TableToProcess candidates.
*
* Tables at risk of wraparound are always processed before routine
* maintenance work. Within each tier, tables are sorted by descending
* urgency score.
*/
static int
TableToProcessComparator(const ListCell *a, const ListCell *b)
{
TableToProcess *ta = (TableToProcess *) lfirst(a);
TableToProcess *tb = (TableToProcess *) lfirst(b);
/* Wraparound prevention always takes priority */
if (ta->wraparound && !tb->wraparound)
return -1;
if (!ta->wraparound && tb->wraparound)
return 1;
/* Within same tier, highest score first */
if (ta->score > tb->score)
return -1;
if (ta->score < tb->score)
return 1;
return 0;
}
The simulation code, workload generator, and visualizations are attached (in "foo_tgz" because my last attempt at this email was stuck in the moderation queue). I'd welcome feedback on whether the scenario is representative. The autovacuum_simulation_fixed.py includes the proposed fix and also runs a number of iteration where it randomized the table Oids so as to remove any dependency on ordering that may be implied in our current algorithm (the before, or v18 algorithm).
The proposed addition to your fix is in the v20260318 patch attached.
best.
-greg
$ ./autovacuum_simulation.py
======================================================================
AUTOVACUUM SCHEDULING SIMULATION (v12 patch)
Accurate model of score-based prioritization
======================================================================
PostgreSQL config:
autovacuum_freeze_max_age = 200,000,000
vacuum_failsafe_age = 1,600,000,000
autovacuum_vacuum_threshold = 50
autovacuum_vacuum_scale_factor= 0.2
autovacuum_max_workers = 3
Simulation: 7 days, 60s steps, seed=42
Generating tables...
5 critical — already past freeze_max_age
15 aging — approaching freeze_max_age
80 active — high dead-tuple rate
BEFORE simulation (catalog OID order):
[OID order ] 0%
[OID order ] 20%
[OID order ] 40%
[OID order ] 60%
[OID order ] 80%
[OID order ] 100%
AFTER simulation (score-based priority):
[Score-based] 0%
[Score-based] 20%
[Score-based] 40%
[Score-based] 60%
[Score-based] 80%
[Score-based] 100%
==============================================================================
RESULTS: Exposure time (minutes at risk before vacuum)
Table OID Crossed Before After Change
------------------------------------------------------------------------------
critical_0 16465 day 0.0 10080m 10080m +0%
critical_1 16398 day 0.0 14m 10080m -71900%
critical_2 16387 day 0.0 4m 10080m -251900%
critical_3 16478 day 0.0 10080m 4m +100%
critical_4 16419 day 0.0 10080m 4m +100%
aging_0 16415 day 0.3 9648m 9648m +0%
aging_1 16412 day 0.7 9114m 9114m +0%
aging_7 16459 day 0.7 9027m 9027m +0%
aging_6 16395 day 0.8 9m 8982m -99700%
aging_10 16481 day 0.9 8721m 8721m +0%
aging_11 16472 day 0.9 8721m 8721m +0%
aging_2 16401 day 1.0 415m 8579m -1967%
aging_3 16397 day 1.0 12m 8578m -71383%
aging_4 16470 day 1.1 8499m 8499m +0%
aging_12 16411 day 1.1 8483m 8483m +0%
aging_8 16438 day 1.2 8296m 8296m +0%
aging_5 16453 day 1.3 8178m 8178m +0%
aging_14 16448 day 2.1 7101m 7101m +0%
aging_9 16388 day 2.2 4m 6979m -174375%
aging_13 16413 day 2.2 6959m 6959m +0%
------------------------------------------------------------------------------
AVERAGE 6172m 7806m -26%
MAXIMUM 10080m 10080m +0%
==============================================================================
Generating visualization...
✓ output/autovacuum_scheduling_impact.png
Done.
-------------------------------------------------------
$ ./autovacuum_simulation_fixed.py
========================================================================
THREE-WAY AUTOVACUUM SCHEDULING COMPARISON
Before (OID) vs v12 Patch (Score) vs Proposed Fix (Tiered)
========================================================================
Config: 3 workers, 7-day sim, 60s steps, 20 runs
Tables: 5 critical + 15 aging + 80 active = 100
freeze_max_age = 200,000,000
Estimated runtime: 3-8 minutes
Run OID avg Score avg Tiered avg
-------------------------------------------
1 7222m 7892m 4m
2 7642m 7892m 4m
3 7961m 7892m 4m
4 6333m 7892m 4m
5 8110m 7892m 4m
6 6359m 7892m 4m
7 6629m 7892m 4m
8 8526m 7892m 4m
9 6385m 7892m 4m
10 6813m 7892m 4m
11 8588m 7892m 4m
12 7261m 7892m 4m
13 6682m 7892m 4m
14 8035m 7892m 4m
15 5667m 7892m 4m
16 7595m 7892m 4m
17 6394m 7892m 4m
18 6686m 7892m 4m
19 7819m 7892m 4m
20 7178m 7892m 4m
========================================================================
AGGREGATE RESULTS
========================================================================
Avg exposure per run (minutes):
OID : 7194 ± 816 (min=5667, max=8588)
Score : 7892 ± 0 (min=7892, max=7892)
Tiered : 4 ± 0 (min=4, max=4)
Peak concurrent force_vacuum tables:
OID : 82 ± 3 (min=79, max=88)
Score : 20 ± 0 (min=20, max=20)
Tiered : 5 ± 0 (min=5, max=5)
Pairwise wins (lower avg exposure = better):
Score beats OID: 5/20 loses: 15/20 ties: 0/20
Tiered beats OID: 20/20 loses: 0/20 ties: 0/20
Tiered beats Score: 20/20 loses: 0/20 ties: 0/20
Variance (std dev of avg exposure across runs):
OID : 816 min
Score : 0 min
Tiered : 0 min
Per-table mean exposure (minutes):
Table OID Score Tiered
--------------------------------------------------------------
critical_0 7564±4470 10080±0 7±0
critical_1 8577±3671 10080±0 7±0
critical_2 9073±3099 8938±0 3±0
critical_3 8581±3661 4±0 4±0
critical_4 9167±2826 3±0 3±0
aging_0 7253±4256 9648±0 4±0
aging_1 7324±3675 9114±0 4±0
aging_2 6865±3517 8579±0 5±0
aging_3 7253±3714 8851±0 4±0
aging_4 6018±4014 8579±0 4±0
aging_5 6856±2951 8064±0 4±0
aging_6 6799±3482 8496±0 3±0
aging_7 5765±4318 8853±0 4±0
aging_8 7091±3227 8644±0 4±0
aging_9 6358±2738 7479±0 4±0
aging_10 7106±3620 8870±0 4±0
aging_11 7175±3673 8965±0 4±0
aging_12 6499±3299 8106±0 4±0
aging_13 6080±3108 7595±0 4±0
aging_14 6479±3913 8891±0 4±0
========================================================================
Completed in 108 seconds (1.8 minutes)
========================================================================
Generating visualization...
✓ output/three_way_comparison.png
Attachments:
[application/octet-stream] v20260318-0001-autovacuum-scheduling-improvements.patch (29.9K, ../../bff7dc5a-f839-449b-b56a-aab36cd3e39d@app.fastmail.com/2-v20260318-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From b907e947e495f717832644f5c740ec75782dc5d9 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v20260318] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 ++++++
doc/src/sgml/maintenance.sgml | 92 ++++++
src/backend/postmaster/autovacuum.c | 263 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 449 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 7c958b06273..b609f05be07 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1061,6 +1061,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 219673db930..53712f80849 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -312,6 +317,13 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+ bool wraparound;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +362,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1867,6 +1880,33 @@ get_database_list(void)
return dblist;
}
+/*
+ * Comparison function for sorting autovac_table candidates.
+ *
+ * Tables at risk of wraparound (force_vacuum) are always processed
+ * before routine maintenance work. Within each tier, tables are
+ * sorted by descending urgency score.
+ */
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *ta = (TableToProcess *) lfirst(a);
+ TableToProcess *tb = (TableToProcess *) lfirst(b);
+
+ /* Wraparound prevention always takes priority */
+ if (ta->wraparound && !tb->wraparound)
+ return -1;
+ if (!ta->wraparound && tb->wraparound)
+ return 1;
+
+ /* Within same tier, highest score first */
+ if (ta->score > tb->score)
+ return -1;
+ if (ta->score < tb->score)
+ return 1;
+ return 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1880,7 +1920,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1992,6 +2032,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2032,11 +2073,19 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ table->wraparound = wraparound;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2092,6 +2141,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2124,11 +2174,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2252,6 +2309,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2280,9 +2339,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2513,7 +2572,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2915,6 +2974,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2922,15 +2982,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2971,6 +3028,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2981,7 +3075,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3010,11 +3105,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ *score = 0.0;
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3062,17 +3162,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3081,13 +3181,58 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3136,34 +3281,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..b47bc534ee1 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '1.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 174e2798443..bc4e6444a2d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3088,6 +3088,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.51.2
[application/octet-stream] foo_tgz (608.7K, ../../bff7dc5a-f839-449b-b56a-aab36cd3e39d@app.fastmail.com/3-foo_tgz)
download
[image/png] three_way_comparison.png (377.2K, ../../bff7dc5a-f839-449b-b56a-aab36cd3e39d@app.fastmail.com/4-three_way_comparison.png)
download | view image
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
@ 2026-03-19 11:44 ` David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2026-03-19 11:44 UTC (permalink / raw)
To: Greg Burd <greg@burd.me>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, 19 Mar 2026 at 22:57, Greg Burd <greg@burd.me> wrote:
> I'm late in the review process. I know David Rowley proposed the unified scoring approach that became the foundation of this patch, and I think that's a great direction. However, I'm concerned that the patch's default scoring weights don't give XID-age urgency sufficient priority over dead-tuple urgency. The weight GUCs (autovacuum_vacuum_score_weight, etc.) can address this, but they max at 1.0, meaning you can only reduce dead-tuple priority, not increase XID priority.
I think that it would be good if you could state *why* you disagree
with the proposed scoring rather than *that* you disagree. All this
stuff was talked about around [1]. For me, I don't see what's
particularly alarming about a table reaching
autovaccum_max_freeze_age. That GUC is set to less than 10% of the
total transaction ID space of where the table must be frozen. Why is
it you think these should take priority over everything else? SLRU
buffers are configurable since v17, so having to lookup the clog for a
wider range of xids isn't as big an issue as it used to be, plus
memory and L3 sizes are bigger than they used to be. Is slow clog
lookups what you're concerned about? You didn't really say.
Having said that, I'd not realised that Nathan capped the new GUCs at
1.0. I think we should allow those to be set higher, likely at least
to 10.0. Maybe we could consider adjusting the code that's setting the
xid_score/mxid_score so that we start scaling the score aggressively
when if (xid_age >= effective_xid_failsafe_age /
Max(autovacuum_freeze_score_weight,1.0)) becomes true. Then, if people
want to play it safer, then they can set
autovacuum_freeze_score_weight = 2.0 and have the aggressive scaling
kick in at 800 million, or whatever half of effective_xid_failsafe_age
is set to. You could set yours to 8.0, if you really want tables over
autovacuum_freeze_max_age to take priority over everything else. I
just don't see or understand the reason why you'd want to.
It's a fairly common misconception that a wraparound vacuum is
something to be alarmed about. Maybe you've fallen for that? I recall
a few proposals to adjust the wording that's shown in pg_stat_activity
to make them seem less alarming.
David
[1] https://www.postgresql.org/message-id/CAApHDvqobtKMwJbhKB_c%3D3-TM%3DTgS3bcuvzcWMm3ee1c0mz9hw%40mail...
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2026-03-19 13:49 ` Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Greg Burd @ 2026-03-19 13:49 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Mar 19, 2026, at 7:44 AM, David Rowley wrote:
> On Thu, 19 Mar 2026 at 22:57, Greg Burd <greg@burd.me> wrote:
>> I'm late in the review process. I know David Rowley proposed the unified scoring approach that became the foundation of this patch, and I think that's a great direction. However, I'm concerned that the patch's default scoring weights don't give XID-age urgency sufficient priority over dead-tuple urgency. The weight GUCs (autovacuum_vacuum_score_weight, etc.) can address this, but they max at 1.0, meaning you can only reduce dead-tuple priority, not increase XID priority.
>
Hello David,
> I think that it would be good if you could state *why* you disagree
> with the proposed scoring rather than *that* you disagree. All this
> stuff was talked about around [1]. For me, I don't see what's
> particularly alarming about a table reaching
> autovaccum_max_freeze_age. That GUC is set to less than 10% of the
> total transaction ID space of where the table must be frozen. Why is
> it you think these should take priority over everything else? SLRU
> buffers are configurable since v17, so having to lookup the clog for a
> wider range of xids isn't as big an issue as it used to be, plus
> memory and L3 sizes are bigger than they used to be. Is slow clog
> lookups what you're concerned about? You didn't really say.
Fair point. Let me be more specific.
My concern isn't that wraparound vacuums are inherently alarming, I agree with you that reaching freeze_max_age isn't a crisis. The issue is a scoring-scale problem in the gap between freeze_max_age (200M) and failsafe age (1.6B).
In that 1.4B XID window, force_vacuum tables have XID scores of 1.0–8.0 (age/freeze_max_age), while typical active tables accumulate dead-tuple scores of 18–70+ within hours of their last vacuum. The exponential boost doesn't activate until failsafe age, so force_vacuum tables are systematically outranked by routine bloat cleanup for what could be days or weeks in production.
I tried to model this in a stress test with 100 tables competing for 3 workers over 7 days, the v12 score-based scheduler actually performed worse than OID order for wraparound exposure:
Algorithm Avg exposure Peak concurrent risk
─────────────────────────────────────────────────
OID 7194 ± 816 min 82 tables
Score 7892 ± 0 min 20 tables
Tiered 4 ± 0 min 5 tables
The score-based scheduler reduced peak concurrent risk (20 vs 82), which is good, but average per-table exposure increased 10% because force_vacuum tables were starved. In 15 out of 20 runs with randomized OIDs, the score scheduler performed worse than OID order.
> Having said that, I'd not realised that Nathan capped the new GUCs at
> 1.0. I think we should allow those to be set higher, likely at least
> to 10.0.
That would definitely help. If autovacuum_freeze_score_weight could be set to 8.0–10.0, DBAs could manually restore the priority we want.
> Maybe we could consider adjusting the code that's setting the
> xid_score/mxid_score so that we start scaling the score aggressively
> when if (xid_age >= effective_xid_failsafe_age /
> Max(autovacuum_freeze_score_weight,1.0)) becomes true
This is clever, it would make the aggressive scaling kick in earlier when the weight is higher. At weight=8.0, you'd get exponential boost starting at 200M (failsafe/8) instead of 1.6B.
Both of these approaches would work. The tiered-sorting proposal was motivated by simplicity. The code already treats wraparound as categorically different (force_vacuum bypasses av_enabled, triggers emergency behavior, can't be disabled per-table). Making the sort order reflect that same categorical distinction felt more aligned with the existing logic than trying to tune scoring weights to create the same effect.
But I'm not religious about it, and I don't have a strong intuition for which would be easier for DBAs to grok and use or for us to maintain so if you do I'll follow your lead. If raising the GUC caps to 10.0 and adding the / Max(weight, 1.0) scaling factor achieves the same goal with less conceptual change, that works for me. The key issue is ensuring force_vacuum tables don't get starved by high-scoring bloat work, and either approach solves that.
> Then, if people
> want to play it safer, then they can set
> autovacuum_freeze_score_weight = 2.0 and have the aggressive scaling
> kick in at 800 million, or whatever half of effective_xid_failsafe_age
> is set to. You could set yours to 8.0, if you really want tables over
> autovacuum_freeze_max_age to take priority over everything else. I
> just don't see or understand the reason why you'd want to.
>
> It's a fairly common misconception that a wraparound vacuum is
> something to be alarmed about. Maybe you've fallen for that?
Not alarmed, but I think the system should process them promptly once they're flagged as force_vacuum, rather than letting them queue behind routine work for potentially days. My simulation suggests the current default weights don't achieve that.
> I recall
> a few proposals to adjust the wording that's shown in pg_stat_activity
> to make them seem less alarming.
>
> David
>
> [1]
> https://www.postgresql.org/message-id/CAApHDvqobtKMwJbhKB_c%3D3-TM%3DTgS3bcuvzcWMm3ee1c0mz9hw%40mail...
Attached is autovacuum_simulation_v3.py which implements your suggestions as a fourth mode called 'dynamic', it has:
- raised GUC caps: the dynamic mode uses weight=8.0
- exponential scaling at: dynamic_xid_threshold = c.effective_xid_failsafe / max(weight, 1.0)
With weight=8.0, this means v12 exponential boost starts at 1.6B XIDs and dynamic starts at 200M XIDs (1.6B / 8.0). Did I capture your suggestion accurately?
I ran the simulation comparing your dynamic scaling approach (weight=8.0, exponential boost at 200M) against tiered sorting. The good news: dynamic scaling is a massive improvement over v12, wraparound exposure is about 38 minutes.
The difference between dynamic and tiered shows up in tables that cross freeze_max_age during the simulation: with score-only sorting, they still compete with high-scoring active tables until their XID ages grow large enough, resulting in wraparound exposures of 24-102 minutes.
Tiered sorting processes them immediately upon crossing the threshold, keeping exposure at 3-5 minutes regardless of when they cross.
Both approaches solve the v12 problem and IMO all three are an improvement over what we ship today. I think some form of this patch should make it into v19.
best.
-greg
$ ./autovacuum_simulation_v3.py
================================================================================
FOUR-WAY AUTOVACUUM SCHEDULING COMPARISON
OID vs v12 Score vs Dynamic Scaling (weight=8.0) vs Tiered Sort
================================================================================
Config: 3 workers, 7-day sim, 60s steps, 20 runs
Tables: 5 critical + 15 aging + 80 active = 100
freeze_max_age = 200,000,000
Estimated runtime: 3-8 minutes
Run OID avg Score avg Dynamic avg Tiered avg
--------------------------------------------------------
1 7222m 7892m 38m 4m
2 7642m 7892m 38m 4m
3 7961m 7892m 38m 4m
4 6333m 7892m 38m 4m
5 8110m 7892m 38m 4m
6 6359m 7892m 38m 4m
7 6629m 7892m 38m 4m
8 8526m 7892m 38m 4m
9 6385m 7892m 38m 4m
10 6813m 7892m 38m 4m
11 8588m 7892m 38m 4m
12 7261m 7892m 38m 4m
13 6682m 7892m 38m 4m
14 8035m 7892m 38m 4m
15 5667m 7892m 38m 4m
16 7595m 7892m 38m 4m
17 6394m 7892m 38m 4m
18 6686m 7892m 38m 4m
19 7819m 7892m 38m 4m
20 7178m 7892m 38m 4m
========================================================================
AGGREGATE RESULTS
========================================================================
Avg exposure per run (minutes):
OID : 7194 ± 816 (min=5667, max=8588)
Score : 7892 ± 0 (min=7892, max=7892)
Dynamic : 38 ± 0 (min=38, max=38)
Tiered : 4 ± 0 (min=4, max=4)
Peak concurrent force_vacuum tables:
OID : 82 ± 3 (min=79, max=88)
Score : 20 ± 0 (min=20, max=20)
Dynamic : 5 ± 0 (min=5, max=5)
Tiered : 5 ± 0 (min=5, max=5)
Pairwise wins (lower avg exposure = better):
Score beats OID: 5/20 loses: 15/20 ties: 0/20
Dynamic beats OID: 20/20 loses: 0/20 ties: 0/20
Tiered beats OID: 20/20 loses: 0/20 ties: 0/20
Dynamic beats Score: 20/20 loses: 0/20 ties: 0/20
Tiered beats Score: 20/20 loses: 0/20 ties: 0/20
Tiered beats Dynamic: 20/20 loses: 0/20 ties: 0/20
Variance (std dev of avg exposure across runs):
OID : 816 min
Score : 0 min
Dynamic : 0 min
Tiered : 0 min
Per-table mean exposure (minutes):
Table OID Score Dynamic Tiered
------------------------------------------------------------------------------
critical_0 7564±4470 10080±0 7±0 7±0
critical_1 8577±3671 10080±0 7±0 7±0
critical_2 9073±3099 8938±0 3±0 3±0
critical_3 8581±3661 4±0 4±0 4±0
critical_4 9167±2826 3±0 3±0 3±0
aging_0 7253±4256 9648±0 95±0 4±0
aging_1 7324±3675 9114±0 24±0 4±0
aging_2 6865±3517 8579±0 86±0 5±0
aging_3 7253±3714 8851±0 18±0 4±0
aging_4 6018±4014 8579±0 47±0 4±0
aging_5 6856±2951 8064±0 90±0 4±0
aging_6 6799±3482 8496±0 45±0 3±0
aging_7 5765±4318 8853±0 21±0 4±0
aging_8 7091±3227 8644±0 6±0 4±0
aging_9 6358±2738 7479±0 102±0 4±0
aging_10 7106±3620 8870±0 36±0 4±0
aging_11 7175±3673 8965±0 65±0 4±0
aging_12 6499±3299 8106±0 51±0 4±0
aging_13 6080±3108 7595±0 28±0 4±0
aging_14 6479±3913 8891±0 27±0 4±0
========================================================================
Completed in 149 seconds (2.5 minutes)
========================================================================
Generating visualization...
✓ output/four_way_comparison.png
Done.
Attachments:
[text/x-python] autovacuum_simulation_v3.py (27.2K, ../../3ca1e398-c787-47e9-9afc-8e298b94dac0@app.fastmail.com/2-autovacuum_simulation_v3.py)
download | inline:
#!/usr/bin/env python3
"""
Four-Way Autovacuum Scheduling Comparison (v12 Patch + Proposed Fixes)
Compares four scheduling strategies across N runs with randomized OID
assignments to verify robustness:
1. Before (OID order) — pre-v19 catalog scan order
2. v12 Patch (Score-based) — unified score, highest first
3. Dynamic Scaling — David Rowley's weight-based early boost
4. Proposed Fix (Tiered sort) — force_vacuum tables always first, then score
Scoring matches the v12 patch (src/backend/postmaster/autovacuum.c):
- XID/MXID age scores: ONLY when force_vacuum (age > freeze_max_age)
- Dead tuple / insert / analyze scores: ONLY when above threshold
- Exponential boost: only past failsafe age (or earlier with dynamic scaling)
- Final score = max(all applicable components)
The dynamic scaling approach (David Rowley's suggestion):
- Allows weights up to 10.0
- Activates exponential boost at effective_failsafe / max(weight, 1.0)
- At weight=8.0, aggressive scaling starts at 200M instead of 1.6B
The tiered sort approach changes only the sort comparator:
sort by (wraparound DESC, score DESC) instead of (score DESC)
This guarantees force_vacuum tables outrank routine maintenance.
Expected results:
- OID: High variance across runs (depends on OID luck)
- Score: Low variance but force_vacuum tables outranked by active tables
- Dynamic: Better prioritization with higher weight
- Tiered: Low variance AND minimal force_vacuum exposure
"""
import copy
import math
import os
import random
import statistics
import time
from dataclasses import dataclass
from typing import Dict, List, Optional
try:
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy as np
HAS_MATPLOTLIB = True
except ImportError:
HAS_MATPLOTLIB = False
print("Warning: matplotlib not available, visualization will be skipped")
# ============================================================================
# Configuration
# ============================================================================
@dataclass
class PgConfig:
autovacuum_freeze_max_age: int = 200_000_000
vacuum_failsafe_age: int = 1_600_000_000
autovacuum_multixact_freeze_max_age: int = 400_000_000
vacuum_multixact_failsafe_age: int = 1_600_000_000
autovacuum_vacuum_threshold: int = 50
autovacuum_vacuum_scale_factor: float = 0.2
autovacuum_vacuum_insert_threshold: int = 1000
autovacuum_vacuum_insert_scale_factor: float = 0.2
autovacuum_analyze_threshold: int = 50
autovacuum_analyze_scale_factor: float = 0.1
autovacuum_freeze_score_weight: float = 1.0
autovacuum_multixact_freeze_score_weight: float = 1.0
autovacuum_vacuum_score_weight: float = 1.0
autovacuum_vacuum_insert_score_weight: float = 1.0
autovacuum_analyze_score_weight: float = 1.0
@property
def effective_xid_failsafe(self) -> int:
return max(self.vacuum_failsafe_age,
int(self.autovacuum_freeze_max_age * 1.05))
@property
def effective_mxid_failsafe(self) -> int:
return max(self.vacuum_multixact_failsafe_age,
int(self.autovacuum_multixact_freeze_max_age * 1.05))
@dataclass
class SimConfig:
num_workers: int = 3
duration_sec: int = 604_800 # 7 days
time_step_sec: int = 60 # 1-minute granularity
vacuum_base_sec: int = 120
vacuum_per_mtuple_sec: int = 30
num_runs: int = 20
master_seed: int = 42
# ============================================================================
# Table & Job
# ============================================================================
@dataclass
class Table:
oid: int
name: str
n_tuples: int
relfrozenxid_age: int
relminmxid_age: int
n_dead_tup: int = 0
n_ins_since_vacuum: int = 0
n_mod_since_analyze: int = 0
xid_rate: int = 1000
dead_rate: int = 0
insert_rate: int = 0
is_being_vacuumed: bool = False
became_force_at: Optional[int] = None
vacuumed_after_force_at: Optional[int] = None
@dataclass
class Job:
table: Table
end_time: int
do_vacuum: bool
do_analyze: bool
# ============================================================================
# Scoring — matches v12 patch (autovacuum.c lines 3161-3290)
# ============================================================================
def relation_needs_vacanalyze(t: Table, c: PgConfig):
"""
Returns (dovacuum, doanalyze, force_vacuum, score).
XID/MXID scores gated on force_vacuum; other scores gated on threshold.
"""
dovacuum = doanalyze = False
score = 0.0
# force_vacuum when age exceeds freeze_max_age
force_vacuum = (t.relfrozenxid_age > c.autovacuum_freeze_max_age or
t.relminmxid_age > c.autovacuum_multixact_freeze_max_age)
# XID/MXID scoring: ONLY inside force_vacuum block
if force_vacuum:
xid_score = t.relfrozenxid_age / c.autovacuum_freeze_max_age
mxid_score = t.relminmxid_age / c.autovacuum_multixact_freeze_max_age
# Exponential boost past failsafe age
if t.relfrozenxid_age >= c.effective_xid_failsafe:
xid_score = math.pow(
xid_score, max(1.0, t.relfrozenxid_age / 100_000_000))
if t.relminmxid_age >= c.effective_mxid_failsafe:
mxid_score = math.pow(
mxid_score, max(1.0, t.relminmxid_age / 100_000_000))
xid_score *= c.autovacuum_freeze_score_weight
mxid_score *= c.autovacuum_multixact_freeze_score_weight
score = max(xid_score, mxid_score)
dovacuum = True
# Dead tuple score: ONLY when above threshold
vacthresh = (c.autovacuum_vacuum_threshold +
c.autovacuum_vacuum_scale_factor * t.n_tuples)
if t.n_dead_tup > vacthresh:
s = (t.n_dead_tup / max(vacthresh, 1)) * c.autovacuum_vacuum_score_weight
score = max(score, s)
dovacuum = True
# Insert score: ONLY when above threshold
insthresh = (c.autovacuum_vacuum_insert_threshold +
c.autovacuum_vacuum_insert_scale_factor * t.n_tuples)
if t.n_ins_since_vacuum > insthresh:
s = (t.n_ins_since_vacuum / max(insthresh, 1)) * c.autovacuum_vacuum_insert_score_weight
score = max(score, s)
dovacuum = True
# Analyze score: ONLY when above threshold
anlthresh = (c.autovacuum_analyze_threshold +
c.autovacuum_analyze_scale_factor * t.n_tuples)
if t.n_mod_since_analyze > anlthresh:
s = (t.n_mod_since_analyze / max(anlthresh, 1)) * c.autovacuum_analyze_score_weight
score = max(score, s)
doanalyze = True
return dovacuum, doanalyze, force_vacuum, score
def relation_needs_vacanalyze_dynamic(t: Table, c: PgConfig, weight: float = 8.0):
"""
Returns (dovacuum, doanalyze, force_vacuum, score).
Implements David Rowley's dynamic scaling suggestion:
- Allows freeze_score_weight up to 10.0
- Activates exponential boost earlier based on weight
- When weight > 1.0, aggressive scaling starts at failsafe / weight
"""
dovacuum = doanalyze = False
score = 0.0
# force_vacuum when age exceeds freeze_max_age
force_vacuum = (t.relfrozenxid_age > c.autovacuum_freeze_max_age or
t.relminmxid_age > c.autovacuum_multixact_freeze_max_age)
# XID/MXID scoring: ONLY inside force_vacuum block
if force_vacuum:
xid_score = t.relfrozenxid_age / c.autovacuum_freeze_max_age
mxid_score = t.relminmxid_age / c.autovacuum_multixact_freeze_max_age
# Dynamic exponential boost based on weight
# Kick in at effective_failsafe / max(weight, 1.0)
dynamic_xid_threshold = c.effective_xid_failsafe / max(weight, 1.0)
dynamic_mxid_threshold = c.effective_mxid_failsafe / max(weight, 1.0)
if t.relfrozenxid_age >= dynamic_xid_threshold:
xid_score = math.pow(
xid_score, max(1.0, t.relfrozenxid_age / 100_000_000))
if t.relminmxid_age >= dynamic_mxid_threshold:
mxid_score = math.pow(
mxid_score, max(1.0, t.relminmxid_age / 100_000_000))
xid_score *= weight
mxid_score *= weight
score = max(xid_score, mxid_score)
dovacuum = True
# Dead tuple score: ONLY when above threshold
vacthresh = (c.autovacuum_vacuum_threshold +
c.autovacuum_vacuum_scale_factor * t.n_tuples)
if t.n_dead_tup > vacthresh:
s = (t.n_dead_tup / max(vacthresh, 1)) * c.autovacuum_vacuum_score_weight
score = max(score, s)
dovacuum = True
# Insert score: ONLY when above threshold
insthresh = (c.autovacuum_vacuum_insert_threshold +
c.autovacuum_vacuum_insert_scale_factor * t.n_tuples)
if t.n_ins_since_vacuum > insthresh:
s = (t.n_ins_since_vacuum / max(insthresh, 1)) * c.autovacuum_vacuum_insert_score_weight
score = max(score, s)
dovacuum = True
# Analyze score: ONLY when above threshold
anlthresh = (c.autovacuum_analyze_threshold +
c.autovacuum_analyze_scale_factor * t.n_tuples)
if t.n_mod_since_analyze > anlthresh:
s = (t.n_mod_since_analyze / max(anlthresh, 1)) * c.autovacuum_analyze_score_weight
score = max(score, s)
doanalyze = True
return dovacuum, doanalyze, force_vacuum, score
# ============================================================================
# Simulation Engine
# ============================================================================
def run_simulation(
tables: List[Table], cfg: PgConfig, scfg: SimConfig,
mode: str, rng: random.Random
) -> Dict:
"""
mode: 'oid' — Before: sort candidates by OID
'score' — v12 Patch: sort candidates by score DESC
'dynamic' — David Rowley: weight=8.0 with early exponential boost
'tiered' — Proposed: sort by (force_vacuum DESC, score DESC)
"""
jobs: List[Job] = []
force_counts: List[int] = []
dt = scfg.time_step_sec
total_steps = scfg.duration_sec // dt
for step in range(total_steps):
now = step * dt
# --- Complete finished jobs ---
finished = [j for j in jobs if j.end_time <= now]
for j in finished:
t = j.table
if j.do_vacuum:
t.n_dead_tup = 0
t.n_ins_since_vacuum = 0
t.relfrozenxid_age = rng.randint(100, 5000)
t.relminmxid_age = rng.randint(50, 2500)
if j.do_analyze:
t.n_mod_since_analyze = 0
t.is_being_vacuumed = False
if (j.do_vacuum
and t.became_force_at is not None
and t.vacuumed_after_force_at is None):
t.vacuumed_after_force_at = now
jobs = [j for j in jobs if j.end_time > now]
# --- Advance workloads ---
for t in tables:
if t.is_being_vacuumed:
continue
t.relfrozenxid_age += t.xid_rate * dt
t.relminmxid_age += (t.xid_rate // 2) * dt
t.n_dead_tup += int(t.dead_rate * dt / 60)
t.n_ins_since_vacuum += int(t.insert_rate * dt / 60)
t.n_mod_since_analyze += int(
(t.dead_rate + t.insert_rate) * dt / 60)
if t.became_force_at is None:
if (t.relfrozenxid_age > cfg.autovacuum_freeze_max_age
or t.relminmxid_age
> cfg.autovacuum_multixact_freeze_max_age):
t.became_force_at = now
# --- Record unvacuumed force_vacuum count ---
fc = sum(1 for t in tables
if not t.is_being_vacuumed
and (t.relfrozenxid_age > cfg.autovacuum_freeze_max_age
or t.relminmxid_age
> cfg.autovacuum_multixact_freeze_max_age))
force_counts.append(fc)
# --- Schedule available workers ---
avail = scfg.num_workers - len(jobs)
if avail > 0:
candidates = []
for t in tables:
if t.is_being_vacuumed:
continue
# Use dynamic scoring for 'dynamic' mode, regular for others
if mode == 'dynamic':
dv, da, fv, sc = relation_needs_vacanalyze_dynamic(t, cfg, weight=8.0)
else:
dv, da, fv, sc = relation_needs_vacanalyze(t, cfg)
if dv or da:
candidates.append((t, sc, dv, da, fv))
# KEY DIFFERENCE between the four modes:
if mode == 'oid':
candidates.sort(key=lambda x: x[0].oid)
elif mode == 'score':
candidates.sort(key=lambda x: x[1], reverse=True)
elif mode == 'dynamic':
# Same as score but with dynamic scaling
candidates.sort(key=lambda x: x[1], reverse=True)
elif mode == 'tiered':
# Wraparound first (True=1 > False=0), then score DESC
candidates.sort(
key=lambda x: (int(x[4]), x[1]), reverse=True)
for t, sc, dv, da, fv in candidates[:avail]:
dur = (scfg.vacuum_base_sec +
int(t.n_tuples / 1_000_000)
* scfg.vacuum_per_mtuple_sec)
t.is_being_vacuumed = True
jobs.append(Job(table=t, end_time=now + dur,
do_vacuum=dv, do_analyze=da))
return {'force_counts': force_counts, 'tables': tables}
# ============================================================================
# Workload Generation
# ============================================================================
RISK_NAMES = ([f'critical_{i}' for i in range(5)] +
[f'aging_{i}' for i in range(15)])
def generate_templates(master_rng: random.Random) -> List[Dict]:
"""
Fixed table properties (no OIDs). OIDs assigned per-run.
5 critical: Already past freeze_max_age (220M-800M).
15 aging: Will cross freeze_max_age at staggered times during sim.
80 active: High dead-tuple generation, keeps workers busy.
"""
freeze_max = 200_000_000
templates: List[Dict] = []
for i, age in enumerate(
[220_000_000, 350_000_000, 500_000_000, 650_000_000, 800_000_000]
):
templates.append(dict(
name=f'critical_{i}',
n_tuples=master_rng.randint(2_000_000, 5_000_000),
relfrozenxid_age=age,
relminmxid_age=age // 3,
xid_rate=master_rng.randint(800, 2000),
dead_rate=master_rng.randint(1, 5),
insert_rate=master_rng.randint(1, 3),
))
for i in range(15):
target_day = 0.3 + (i / 14) * 5.2
target_sec = int(target_day * 86400)
rate = master_rng.randint(800, 2500)
start_age = max(50_000_000, freeze_max - rate * target_sec)
templates.append(dict(
name=f'aging_{i}',
n_tuples=master_rng.randint(2_000_000, 5_000_000),
relfrozenxid_age=start_age,
relminmxid_age=start_age // 3,
xid_rate=rate,
dead_rate=master_rng.randint(1, 5),
insert_rate=master_rng.randint(1, 3),
))
for i in range(80):
n_tup = master_rng.randint(100_000, 2_000_000)
vacthresh = 50 + 0.2 * n_tup
templates.append(dict(
name=f'active_{i}',
n_tuples=n_tup,
relfrozenxid_age=master_rng.randint(10_000_000, 80_000_000),
relminmxid_age=master_rng.randint(5_000_000, 40_000_000),
n_dead_tup=master_rng.randint(
int(vacthresh * 0.3), int(vacthresh * 2.5)),
xid_rate=master_rng.randint(500, 1500),
dead_rate=master_rng.randint(3000, 12000),
insert_rate=master_rng.randint(1000, 5000),
))
return templates
def instantiate_tables(
templates: List[Dict], oid_rng: random.Random
) -> List[Table]:
"""Create Table objects with freshly shuffled OIDs."""
oids = list(range(16384, 16384 + len(templates)))
oid_rng.shuffle(oids)
tables = []
stat_keys = ('n_dead_tup', 'n_ins_since_vacuum', 'n_mod_since_analyze')
for tmpl, oid in zip(templates, oids):
rest = {k: v for k, v in tmpl.items() if k not in stat_keys}
tables.append(Table(
oid=oid,
n_dead_tup=tmpl.get('n_dead_tup', 0),
n_ins_since_vacuum=tmpl.get('n_ins_since_vacuum', 0),
n_mod_since_analyze=tmpl.get('n_mod_since_analyze', 0),
**rest
))
return tables
def compute_exposures(tables: List[Table], duration: int) -> Dict[str, float]:
"""Return {table_name: exposure_minutes} for at-risk tables."""
tmap = {t.name: t for t in tables}
out = {}
for name in RISK_NAMES:
t = tmap.get(name)
if t and t.became_force_at is not None:
end = (t.vacuumed_after_force_at
if t.vacuumed_after_force_at is not None
else duration)
out[name] = (end - t.became_force_at) / 60.0
return out
def safe_stdev(vals):
"""stdev that handles < 2 values."""
if len(vals) < 2:
return 0.0
return statistics.stdev(vals)
# ============================================================================
# Visualization
# ============================================================================
MODES = ['oid', 'score', 'dynamic', 'tiered']
MODE_LABELS = {
'oid': 'Before (OID Order)',
'score': 'v12 Patch (Score)',
'dynamic': 'Dynamic (weight=8.0)',
'tiered': 'Proposed (Tiered)',
}
MODE_SHORT = {'oid': 'OID', 'score': 'Score', 'dynamic': 'Dynamic', 'tiered': 'Tiered'}
COLORS = {'oid': '#d62728', 'score': '#1f77b4', 'dynamic': '#ff7f0e', 'tiered': '#2ca02c'}
def create_visualization(data, scfg, cfg):
if not HAS_MATPLOTLIB:
return
os.makedirs('output', exist_ok=True)
n_runs = scfg.num_runs
total_steps = scfg.duration_sec // scfg.time_step_sec
time_days = np.arange(total_steps) * scfg.time_step_sec / 86400.0
fig = plt.figure(figsize=(20, 15))
gs = fig.add_gridspec(3, 1, hspace=0.3)
# ================================================================
# Panel 1: All Four Modes
# ================================================================
ax1 = fig.add_subplot(gs[0])
for mode in MODES:
tl = data['force_timelines'][mode]
min_len = min(len(t) for t in tl) if tl else 0
if min_len > 0:
arr = np.array([t[:min_len] for t in tl])
td = time_days[:min_len]
mean_line = arr.mean(axis=0)
std_line = arr.std(axis=0)
ax1.plot(td, mean_line, color=COLORS[mode], linewidth=2.5,
label=MODE_LABELS[mode])
ax1.fill_between(td,
np.maximum(mean_line - std_line, 0),
mean_line + std_line,
color=COLORS[mode], alpha=0.12)
ax1.set_xlabel('Time (days)', fontsize=12)
ax1.set_ylabel('Unvacuumed Tables Past freeze_max_age', fontsize=12)
ax1.set_title(
'Panel 1: All Four Scheduling Strategies\n'
f'(mean ± std, {n_runs} runs with randomized OIDs)',
fontsize=14, fontweight='bold')
ax1.legend(fontsize=11, loc='upper right')
ax1.set_xlim(0, scfg.duration_sec / 86400)
ax1.set_ylim(bottom=-0.5)
ax1.grid(True, alpha=0.3)
# ================================================================
# Panel 2: v12, Dynamic, Tiered (without baseline)
# ================================================================
ax2 = fig.add_subplot(gs[1])
modes_subset = ['score', 'dynamic', 'tiered']
for mode in modes_subset:
tl = data['force_timelines'][mode]
min_len = min(len(t) for t in tl) if tl else 0
if min_len > 0:
arr = np.array([t[:min_len] for t in tl])
td = time_days[:min_len]
mean_line = arr.mean(axis=0)
std_line = arr.std(axis=0)
ax2.plot(td, mean_line, color=COLORS[mode], linewidth=2.5,
label=MODE_LABELS[mode])
ax2.fill_between(td,
np.maximum(mean_line - std_line, 0),
mean_line + std_line,
color=COLORS[mode], alpha=0.12)
ax2.set_xlabel('Time (days)', fontsize=12)
ax2.set_ylabel('Unvacuumed Tables Past freeze_max_age', fontsize=12)
ax2.set_title(
'Panel 2: Comparing Proposed Fixes to v12 Baseline\n'
f'(v12 Score vs Dynamic Scaling vs Tiered Sort)',
fontsize=14, fontweight='bold')
ax2.legend(fontsize=11, loc='upper right')
ax2.set_xlim(0, scfg.duration_sec / 86400)
ax2.set_ylim(bottom=-0.5)
ax2.grid(True, alpha=0.3)
# ================================================================
# Panel 3: Dynamic vs Tiered only
# ================================================================
ax3 = fig.add_subplot(gs[2])
modes_final = ['dynamic', 'tiered']
for mode in modes_final:
tl = data['force_timelines'][mode]
min_len = min(len(t) for t in tl) if tl else 0
if min_len > 0:
arr = np.array([t[:min_len] for t in tl])
td = time_days[:min_len]
mean_line = arr.mean(axis=0)
std_line = arr.std(axis=0)
ax3.plot(td, mean_line, color=COLORS[mode], linewidth=3.0,
label=MODE_LABELS[mode])
ax3.fill_between(td,
np.maximum(mean_line - std_line, 0),
mean_line + std_line,
color=COLORS[mode], alpha=0.15)
ax3.set_xlabel('Time (days)', fontsize=12)
ax3.set_ylabel('Unvacuumed Tables Past freeze_max_age', fontsize=12)
ax3.set_title(
'Panel 3: Direct Comparison of Proposed Fixes\n'
f'(Dynamic Scaling weight=8.0 vs Tiered Sort)',
fontsize=14, fontweight='bold')
ax3.legend(fontsize=11, loc='upper right')
ax3.set_xlim(0, scfg.duration_sec / 86400)
ax3.set_ylim(bottom=-0.5)
ax3.grid(True, alpha=0.3)
fig.suptitle(
'PostgreSQL 19: Autovacuum Scheduling — Wraparound Risk Over Time\n'
'Comparing v12 Patch, Dynamic Scaling (weight=8.0), and Tiered Sort',
fontsize=16, fontweight='bold', y=0.995)
plt.savefig('output/wraparound_risk_comparison.png',
dpi=150, bbox_inches='tight')
plt.close()
print(" ✓ output/wraparound_risk_comparison.png")
# ============================================================================
# Main
# ============================================================================
def main():
wall_start = time.time()
print("=" * 80)
print("FOUR-WAY AUTOVACUUM SCHEDULING COMPARISON")
print("OID vs v12 Score vs Dynamic Scaling (weight=8.0) vs Tiered Sort")
print("=" * 80)
cfg = PgConfig()
scfg = SimConfig()
print(f"\nConfig: {scfg.num_workers} workers, "
f"{scfg.duration_sec // 86400}-day sim, "
f"{scfg.time_step_sec}s steps, "
f"{scfg.num_runs} runs")
print("Tables: 5 critical + 15 aging + 80 active = 100")
print(f"freeze_max_age = {cfg.autovacuum_freeze_max_age:,}")
print(f"Estimated runtime: 3-8 minutes\n")
master_rng = random.Random(scfg.master_seed)
templates = generate_templates(master_rng)
# Accumulators
data: Dict = {
'force_timelines': {m: [] for m in MODES},
'avg_exposures': {m: [] for m in MODES},
'per_table': {m: {n: [] for n in RISK_NAMES} for m in MODES},
'peaks': {m: [] for m in MODES},
}
hdr = f"{'Run':>4}"
for m in MODES:
hdr += f" {MODE_SHORT[m] + ' avg':>11}"
print(hdr)
print("-" * (4 + 13 * len(MODES)))
for run_idx in range(scfg.num_runs):
run_seed = 1000 + run_idx
base_tables = instantiate_tables(templates, random.Random(run_seed))
line = f" {run_idx + 1:>2}"
for mi, mode in enumerate(MODES):
tables = copy.deepcopy(base_tables)
rng = random.Random(run_seed + 5000 + mi * 1000)
res = run_simulation(tables, cfg, scfg, mode, rng)
fc = res['force_counts']
data['force_timelines'][mode].append(fc)
data['peaks'][mode].append(max(fc) if fc else 0)
exp = compute_exposures(res['tables'], scfg.duration_sec)
vals = list(exp.values())
avg = statistics.mean(vals) if vals else 0.0
data['avg_exposures'][mode].append(avg)
for name in RISK_NAMES:
if name in exp:
data['per_table'][mode][name].append(exp[name])
line += f" {avg:>9.0f}m"
print(line)
# ================================================================
# Aggregate Results
# ================================================================
print("\n" + "=" * 72)
print("AGGREGATE RESULTS")
print("=" * 72)
def fmts(vals):
if not vals:
return "n/a"
m = statistics.mean(vals)
s = safe_stdev(vals)
return (f"{m:>7.0f} ± {s:<7.0f} "
f"(min={min(vals):.0f}, max={max(vals):.0f})")
print(f"\nAvg exposure per run (minutes):")
for mode in MODES:
print(f" {MODE_SHORT[mode]:<10}: "
f"{fmts(data['avg_exposures'][mode])}")
print(f"\nPeak concurrent force_vacuum tables:")
for mode in MODES:
print(f" {MODE_SHORT[mode]:<10}: "
f"{fmts([float(x) for x in data['peaks'][mode]])}")
print(f"\nPairwise wins (lower avg exposure = better):")
n = scfg.num_runs
pairs = [('oid', 'score'), ('oid', 'dynamic'), ('oid', 'tiered'),
('score', 'dynamic'), ('score', 'tiered'), ('dynamic', 'tiered')]
for m1, m2 in pairs:
w = sum(1 for a, b in zip(
data['avg_exposures'][m1], data['avg_exposures'][m2]) if b < a)
l = sum(1 for a, b in zip(
data['avg_exposures'][m1], data['avg_exposures'][m2]) if b > a)
t = n - w - l
print(f" {MODE_SHORT[m2]} beats {MODE_SHORT[m1]}: "
f"{w}/{n} loses: {l}/{n} ties: {t}/{n}")
print(f"\nVariance (std dev of avg exposure across runs):")
for mode in MODES:
s = safe_stdev(data['avg_exposures'][mode])
print(f" {MODE_SHORT[mode]:<10}: {s:.0f} min")
print(f"\nPer-table mean exposure (minutes):")
hdr2 = f" {'Table':<14}"
for m in MODES:
hdr2 += f" {MODE_SHORT[m]:>14}"
print(hdr2)
print(" " + "-" * (14 + 16 * len(MODES)))
for name in RISK_NAMES:
line = f" {name:<14}"
for mode in MODES:
vals = data['per_table'][mode].get(name, [])
if vals:
m = statistics.mean(vals)
s = safe_stdev(vals)
line += f" {m:>6.0f}±{s:<5.0f}"
else:
line += f" {'n/a':>14}"
print(line)
elapsed = time.time() - wall_start
print(f"\n{'=' * 72}")
print(f"Completed in {elapsed:.0f} seconds ({elapsed / 60:.1f} minutes)")
print(f"{'=' * 72}")
# ================================================================
# Visualization
# ================================================================
if HAS_MATPLOTLIB:
print("\nGenerating visualization...")
create_visualization(data, scfg, cfg)
else:
print("\nSkipping visualization (matplotlib not installed).")
print("\nDone.")
if __name__ == '__main__':
main()
[image/png] wraparound_risk_comparison.png (377.4K, ../../3ca1e398-c787-47e9-9afc-8e298b94dac0@app.fastmail.com/3-wraparound_risk_comparison.png)
download | view image
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
@ 2026-03-19 15:49 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 17:36 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Nathan Bossart @ 2026-03-19 15:49 UTC (permalink / raw)
To: Greg Burd <greg@burd.me>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Mar 19, 2026 at 09:49:34AM -0400, Greg Burd wrote:
> My concern isn't that wraparound vacuums are inherently alarming, I agree
> with you that reaching freeze_max_age isn't a crisis. The issue is a
> scoring-scale problem in the gap between freeze_max_age (200M) and
> failsafe age (1.6B).
>
> In that 1.4B XID window, force_vacuum tables have XID scores of 1.0–8.0
> (age/freeze_max_age), while typical active tables accumulate dead-tuple
> scores of 18–70+ within hours of their last vacuum. The exponential boost
> doesn't activate until failsafe age, so force_vacuum tables are
> systematically outranked by routine bloat cleanup for what could be days
> or weeks in production.
I think "systematically outranked" makes the problem sound worse than it
is. Once the freeze age is reached, the table is going to get added to the
list no matter what, it just might be sorted lower.
>> Having said that, I'd not realised that Nathan capped the new GUCs at
>> 1.0. I think we should allow those to be set higher, likely at least
>> to 10.0.
>
> That would definitely help. If autovacuum_freeze_score_weight could be
> set to 8.0–10.0, DBAs could manually restore the priority we want.
Done in the attached.
>> Maybe we could consider adjusting the code that's setting the
>> xid_score/mxid_score so that we start scaling the score aggressively
>> when if (xid_age >= effective_xid_failsafe_age /
>> Max(autovacuum_freeze_score_weight,1.0)) becomes true
>
> This is clever, it would make the aggressive scaling kick in earlier when
> the weight is higher. At weight=8.0, you'd get exponential boost starting
> at 200M (failsafe/8) instead of 1.6B.
Seems reasonable. I've added this, too. Something else we might want to
consider is scaling the score once the freeze age is reached, just much
less aggressively than we do at the failsafe age. It probably doesn't make
sense to start scaling too much at 200M, but at 1.5B, yeah, we should
probably process the table sooner than later.
--
nathan
From 335b718302be1ae285145e0f55924585a9d91b8e Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v13 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
doc/src/sgml/maintenance.sgml | 92 +++++++
src/backend/postmaster/autovacuum.c | 252 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 438 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 7c958b06273..b609f05be07 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1061,6 +1061,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 219673db930..55d3adc2909 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -312,6 +317,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +361,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1867,6 +1879,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1880,7 +1901,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1992,6 +2013,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2032,11 +2054,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2092,6 +2121,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2124,11 +2154,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2252,6 +2289,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2280,9 +2319,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2513,7 +2552,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2915,6 +2954,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2922,15 +2962,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2971,6 +3008,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2981,7 +3055,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3010,11 +3085,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ *score = 0.0;
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3062,17 +3142,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3081,13 +3161,67 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * being scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3136,34 +3270,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 4673eca9cd6..b89ae469ddd 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3088,6 +3088,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v13-0001-autovacuum-scheduling-improvements.patch (29.9K, ../../abwbDTu6OO6Izp4o@nathan/2-v13-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From 335b718302be1ae285145e0f55924585a9d91b8e Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v13 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
doc/src/sgml/maintenance.sgml | 92 +++++++
src/backend/postmaster/autovacuum.c | 252 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 438 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 7c958b06273..b609f05be07 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1061,6 +1061,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 219673db930..55d3adc2909 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -312,6 +317,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +361,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1867,6 +1879,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1880,7 +1901,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1992,6 +2013,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2032,11 +2054,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2092,6 +2121,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2124,11 +2154,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2252,6 +2289,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2280,9 +2319,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2513,7 +2552,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2915,6 +2954,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2922,15 +2962,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2971,6 +3008,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2981,7 +3055,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3010,11 +3085,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ *score = 0.0;
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3062,17 +3142,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3081,13 +3161,67 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * being scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3136,34 +3270,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 4673eca9cd6..b89ae469ddd 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3088,6 +3088,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-19 17:36 ` Greg Burd <greg@burd.me>
1 sibling, 0 replies; 150+ messages in thread
From: Greg Burd @ 2026-03-19 17:36 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Mar 19, 2026, at 11:49 AM, Nathan Bossart wrote:
> On Thu, Mar 19, 2026 at 09:49:34AM -0400, Greg Burd wrote:
>> My concern isn't that wraparound vacuums are inherently alarming, I agree
>> with you that reaching freeze_max_age isn't a crisis. The issue is a
>> scoring-scale problem in the gap between freeze_max_age (200M) and
>> failsafe age (1.6B).
>>
>> In that 1.4B XID window, force_vacuum tables have XID scores of 1.0–8.0
>> (age/freeze_max_age), while typical active tables accumulate dead-tuple
>> scores of 18–70+ within hours of their last vacuum. The exponential boost
>> doesn't activate until failsafe age, so force_vacuum tables are
>> systematically outranked by routine bloat cleanup for what could be days
>> or weeks in production.
>
> I think "systematically outranked" makes the problem sound worse than it
> is. Once the freeze age is reached, the table is going to get added to the
> list no matter what, it just might be sorted lower.
Yeah, that was a bit of hyperbole on my part. :)
>>> Having said that, I'd not realised that Nathan capped the new GUCs at
>>> 1.0. I think we should allow those to be set higher, likely at least
>>> to 10.0.
>>
>> That would definitely help. If autovacuum_freeze_score_weight could be
>> set to 8.0–10.0, DBAs could manually restore the priority we want.
>
> Done in the attached.
+1
>>> Maybe we could consider adjusting the code that's setting the
>>> xid_score/mxid_score so that we start scaling the score aggressively
>>> when if (xid_age >= effective_xid_failsafe_age /
>>> Max(autovacuum_freeze_score_weight,1.0)) becomes true
>>
>> This is clever, it would make the aggressive scaling kick in earlier when
>> the weight is higher. At weight=8.0, you'd get exponential boost starting
>> at 200M (failsafe/8) instead of 1.6B.
>
> Seems reasonable. I've added this, too.
+1
> Something else we might want to
> consider is scaling the score once the freeze age is reached, just much
> less aggressively than we do at the failsafe age. It probably doesn't make
> sense to start scaling too much at 200M, but at 1.5B, yeah, we should
> probably process the table sooner than later.
So a scaling factor relative to some point like 200M? Maybe... but for now I think what you have in v13 is about right and a solid improvement over what's there now.
> --
> nathan
>
> Attachments:
> * v13-0001-autovacuum-scheduling-improvements.patch
LGTM!
best.
-greg
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-20 04:22 ` Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2026-03-20 04:22 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Greg Burd <greg@burd.me>; David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> Done in the attached.
Thanks!
I have a few comments.
#1.
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
This should be "0.0 and 10.0"
#2.
typo:
+ * being scaling aggressively.
Thisshould be "begin"
#3.
This reads more like documentation than code comment.
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 1.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
I don't actually think this section adds any value at all to the comments.
#4.
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f),
ins: (disabled), anl: %.0f (threshold %.0f), score %.3f",
A missing colon after "score", unlike the other occurrence which has it.
#5.
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /=
autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /=
autovacuum_multixact_freeze_score_weight;
+
Shouldn't this be "if (autovacuum_freeze_score_weight > 0.0)" ?
A weight > 0 should always adjust the threshold, right? we should only
prevent division by 0 here.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2026-03-20 04:40 ` David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2026-03-20 04:40 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, 20 Mar 2026 at 17:22, Sami Imseih <samimseih@gmail.com> wrote:
> #5.
> + if (autovacuum_freeze_score_weight > 1.0)
> + effective_xid_failsafe_age /=
> autovacuum_freeze_score_weight;
> + if (autovacuum_multixact_freeze_score_weight > 1.0)
> + effective_mxid_failsafe_age /=
> autovacuum_multixact_freeze_score_weight;
> +
>
> Shouldn't this be "if (autovacuum_freeze_score_weight > 0.0)" ?
> A weight > 0 should always adjust the threshold, right? we should only
> prevent division by 0 here.
We really do want to ensure that tables are scaling very aggressively
when they reach the failsafe age. We don't want any quirky user
settings changing that. Prioritising anything else over a table at
failsafe age would be a very bad thing. The only point in doing what I
suggested was to allow users to give themselves more margin to get the
freezing done before failsafe age, certainly not less margin. In any
case, effective_xid_failsafe_age and effective_mxid_failsafe_age are
signed ints and default to 1.6 billion. There's just not enough
bit-space to divide them by any number much below 1.0 before they'll
wrap.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2026-03-20 05:03 ` Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2026-03-20 05:03 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
>
> case, effective_xid_failsafe_age and effective_mxid_failsafe_age are
> signed ints and default to 1.6 billion. There's just not enough
> bit-space to divide them by any number much below 1.0 before they'll
> wrap.
>
Of course. It Would not be safe to divide by < 1.
Thanks!
--
Sami
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2026-03-20 20:31 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-21 14:55 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Nathan Bossart @ 2026-03-20 20:31 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Thanks for the feedback. Here is an updated patch.
I kept the comment about the weight parameters in autovacuum.c. Since
there's a bunch of code related to them, IMHO we should have an explanatory
note somewhere.
--
nathan
From 0d2fd4d3af32656e574bf81189cb2ac536f97b26 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v14 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
doc/src/sgml/maintenance.sgml | 92 +++++++
src/backend/postmaster/autovacuum.c | 252 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 438 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 75c22405a09..b5a191c130b 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1062,6 +1062,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 219673db930..b5c153a8835 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -312,6 +317,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +361,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1867,6 +1879,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1880,7 +1901,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1992,6 +2013,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2032,11 +2054,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2092,6 +2121,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2124,11 +2154,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2252,6 +2289,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2280,9 +2319,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2513,7 +2552,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2915,6 +2954,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2922,15 +2962,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2971,6 +3008,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 10.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2981,7 +3055,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3010,11 +3085,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ *score = 0.0;
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3062,17 +3142,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3081,13 +3161,67 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * begin scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3136,34 +3270,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 0042c33fa66..a8da75c4396 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3094,6 +3094,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v14-0001-autovacuum-scheduling-improvements.patch (29.9K, ../../ab2uoj5yU8CVUJ_H@nathan/2-v14-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From 0d2fd4d3af32656e574bf81189cb2ac536f97b26 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v14 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 +++++++
doc/src/sgml/maintenance.sgml | 92 +++++++
src/backend/postmaster/autovacuum.c | 252 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 438 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 75c22405a09..b5a191c130b 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1062,6 +1062,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 219673db930..b5c153a8835 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -312,6 +317,12 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +361,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ double *score);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1867,6 +1879,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1880,7 +1901,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1992,6 +2013,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2032,11 +2054,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2092,6 +2121,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ double score = 0.0;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2124,11 +2154,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &score);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = score;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2252,6 +2289,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2280,9 +2319,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2513,7 +2552,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2915,6 +2954,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ double score;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2922,15 +2962,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &score);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2971,6 +3008,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 10.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2981,7 +3055,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ double *score)
{
bool force_vacuum;
bool av_enabled;
@@ -3010,11 +3085,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ *score = 0.0;
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3062,17 +3142,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3081,13 +3161,67 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ double xid_score;
+ double mxid_score;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ xid_score = (double) xid_age / freeze_max_age;
+ mxid_score = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * begin scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ xid_score = pow(xid_score, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ mxid_score = pow(mxid_score, Max(1.0, (double) mxid_age / 100000000));
+
+ xid_score *= autovacuum_freeze_score_weight;
+ mxid_score *= autovacuum_multixact_freeze_score_weight;
+
+ *score = Max(xid_score, mxid_score);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3136,34 +3270,58 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ double vacthresh_score;
+
+ vacthresh_score = (double) vactuples / Max(vacthresh, 1);
+ vacthresh_score *= autovacuum_vacuum_score_weight;
+
+ *score = Max(*score, vacthresh_score);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ double vacinsthresh_score;
+
+ vacinsthresh_score = (double) instuples / Max(vacinsthresh, 1);
+ vacinsthresh_score *= autovacuum_vacuum_insert_score_weight;
+
+ *score = Max(*score, vacinsthresh_score);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ double anlthresh_score;
+
+ anlthresh_score = (double) anltuples / Max(anlthresh, 1);
+ anlthresh_score *= autovacuum_analyze_score_weight;
+
+ *score = Max(*score, anlthresh_score);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score: %.3f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ vactuples, vacthresh, anltuples, anlthresh, *score);
}
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 0042c33fa66..a8da75c4396 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3094,6 +3094,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-21 14:55 ` Sami Imseih <samimseih@gmail.com>
1 sibling, 0 replies; 150+ messages in thread
From: Sami Imseih @ 2026-03-21 14:55 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> Thanks for the feedback. Here is an updated patch.
LGTM.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-22 00:08 ` Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Bharath Rupireddy @ 2026-03-22 00:08 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, Mar 20, 2026 at 1:31 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> Thanks for the feedback. Here is an updated patch.
>
> I kept the comment about the weight parameters in autovacuum.c. Since
> there's a bunch of code related to them, IMHO we should have an explanatory
> note somewhere.
Hi Nathan, Thank you for working on this feature. I'm late to this
thread. I read the patch, and here are some general thoughts. I
haven't read the full thread, so some of these thoughts may be
repetitive - thanks for bearing with me.
1/ The Autovacuum Prioritization section in the docs is a good start
for explaining the usage aspects of the new scoring system. However,
IMHO, adding a couple of production-like scenarios showing how these
scores need to be adjusted and used would be a good addition to this
section.
2/ Any plans to extend the new scoring system to the table level
(i.e., reloptions)? I think it would help in situations where there
are huge tables that need to be prioritized for vacuum over others, so
setting the scoring high for those tables would allow the next
autovacuum to pick them up first.
3/ Any plans to add tests to demo how each of these parameters could
help in various situations - when the system needs freezing to be
prioritized for avoiding XID wraparound over cleaning up dead tuples,
and when it needs analyze to be prioritized to get correct plans,
etc.? If needed, we could add elog(DEBUGX) messages to emit the reason
or effectiveness of these new scores so that we can verify them in
tests.
4/ Is adding a reason (such as how each of these scores influenced the
autovacuum to pick this table) to vacuum progress reporting a good
idea? This helps answer some of the why and how questions when the
autovacuum is in progress.
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
@ 2026-03-23 15:22 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-23 15:22 UTC (permalink / raw)
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sat, Mar 21, 2026 at 05:08:56PM -0700, Bharath Rupireddy wrote:
> Hi Nathan, Thank you for working on this feature. I'm late to this
> thread. I read the patch, and here are some general thoughts. I
> haven't read the full thread, so some of these thoughts may be
> repetitive - thanks for bearing with me.
Thanks for looking.
> 1/ The Autovacuum Prioritization section in the docs is a good start
> for explaining the usage aspects of the new scoring system. However,
> IMHO, adding a couple of production-like scenarios showing how these
> scores need to be adjusted and used would be a good addition to this
> section.
IMO these scores generally shouldn't need adjusting outside of cases where
the DBA wants to revert to the previous scheduling strategy or there's some
other unique scenario where the prioritization isn't working for them. I
think it'd be hard to give any general guidance, at least at this point
since we have no field experience yet.
> 2/ Any plans to extend the new scoring system to the table level
> (i.e., reloptions)? I think it would help in situations where there
> are huge tables that need to be prioritized for vacuum over others, so
> setting the scoring high for those tables would allow the next
> autovacuum to pick them up first.
I thought about that, but decided to leave those out because 1) the scoring
parameters are meant to be global and affect workers' prioritization of all
tables and 2) there are already a number of parameters that can be adjusted
to affect the score. If users find that they really need reloptions for
some reason, we can always add them later. Removing them seems more
difficult.
> 3/ Any plans to add tests to demo how each of these parameters could
> help in various situations - when the system needs freezing to be
> prioritized for avoiding XID wraparound over cleaning up dead tuples,
> and when it needs analyze to be prioritized to get correct plans,
> etc.? If needed, we could add elog(DEBUGX) messages to emit the reason
> or effectiveness of these new scores so that we can verify them in
> tests.
I hadn't planned on doing any additional tests/demonstrations here. And
I'd really like to avoid burying all this information in DEBUG log
messages. IMO we ought to eventually be exposing this stuff in system
views and the like, as has been discussed elsewhere.
> 4/ Is adding a reason (such as how each of these scores influenced the
> autovacuum to pick this table) to vacuum progress reporting a good
> idea? This helps answer some of the why and how questions when the
> autovacuum is in progress.
Yeah, adding that in addition to a system view, etc. could be nice. I'm a
little hesitant to start making big additions to the patch at this point,
but I can give it a whirl if folks think something like this should be
added for v19.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-23 19:01 ` Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2026-03-23 19:01 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> > 4/ Is adding a reason (such as how each of these scores influenced the
> > autovacuum to pick this table) to vacuum progress reporting a good
> > idea? This helps answer some of the why and how questions when the
> > autovacuum is in progress.
>
> Yeah, adding that in addition to a system view, etc. could be nice. I'm a
> little hesitant to start making big additions to the patch at this point,
> but I can give it a whirl if folks think something like this should be
> added for v19.
Adding a system view will be nice. I am attaching a version I used in earlier
testing (cleaned up with docs), if we are inclined to get this in. I
think it will be
useful.
This follows the same setup as do_autovacuum(); scanning pg_class,
filtering relation kinds and temp tables, and computing
effective_multixact_freeze_max_age
are done in the SQL-callable function, while another wrapper
compute_autovac_score() handles
the per-relation setup (snapshotting recentXid/recentMulti, fetching
reloptions and the pgstat entry)
before calling relation_needs_vacanalyze(). The function holds an
AccessShareLock on pg_class for
the duration of the scan, so this should be relatively lightweight.
```
test=# select * from pg_stat_autovacuum_priority order by score desc ;
relid | schemaname | relname | dovacuum |
doanalyze | wraparound | score
-------+--------------------+-----------------------------+----------+-----------+------------+-----------------------
16400 | public | pgbench_accounts | t |
f | t | 1.055318563196673e+16
16404 | public | pgbench_branches | t |
t | t | 442.01666666666665
16396 | public | pgbench_tellers | t |
t | t | 172.97333333333333
16393 | public | pgbench_history | t |
t | t | 4.703261221642761
14227 | pg_toast | pg_toast_14224 | t |
f | t | 2.08555407
```
Note in the test above, I used xid_wraparound to calculate a score
with the failsafe POW()
adjustment. Notice that this is a very high score being emitted as
discussed earlier [1].
This is documented in v14 as "scaled aggressively so that the table
has a decent chance of
sorting to the top of the list."
Maybe the doc should say something like " scaled aggressively, which
can produce very large values, to ensure
the table sorts to the top of the list."
[1] [https://www.postgresql.org/message-id/CAA5RZ0vfhAnFBp4HrBQc%2BALaJMx6vCvMtnBi39ST_4nH9PZEjA%40mail.g...]
--
Sami Imseih
Amazon Web Services (AWS)
Attachments:
[application/octet-stream] v1-0001-Add-pg_stat_autovacuum_priority-view.patch (15.3K, ../../CAA5RZ0tz55uoQfcvLwvRWRPM0ry6SGfZ=-enQ4F0kaUgfPh+Vg@mail.gmail.com/2-v1-0001-Add-pg_stat_autovacuum_priority-view.patch)
download | inline diff:
From 2e08f022fef177c586838d1adb3f14202ea48578 Mon Sep 17 00:00:00 2001
From: Sami Imseih <samimseih@gmail.com>
Date: Mon, 23 Mar 2026 17:03:59 +0000
Subject: [PATCH v1 1/1] Add pg_stat_autovacuum_priority view
Add a new system view that exposes the autovacuum
priority score for each relation in the current
database. This allows users to inspect each table's
autovacuum eligibility and priority.
The columns returned are: relid, schemaname, relname,
needs_vacuum, needs_analyze, wraparound, and score.
The view results are based on the output of
relation_needs_vacanalyze(), in which the same setup
as do_autovacuum() is performed before calling
relation_needs_vacanalyze(). pg_class is scanned with
an AccessShareLock, so it is relatively lightweight.
Unlike do_autovacuum(), we don't need to derive
pg_toast relationships to the relation in advance, and
we just treat TOAST tables as another relation coming
in from pg_class.
---
doc/src/sgml/maintenance.sgml | 6 ++
doc/src/sgml/monitoring.sgml | 108 ++++++++++++++++++++++
src/backend/catalog/system_views.sql | 13 +++
src/backend/postmaster/autovacuum.c | 132 +++++++++++++++++++++++++--
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 9 ++
src/test/regress/expected/rules.out | 10 ++
7 files changed, 270 insertions(+), 10 deletions(-)
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index b5a191c130b..1a262fa1244 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1153,6 +1153,12 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
listed in the <literal>pg_class</literal> system catalog), set all of the
aforementioned "weight" parameters to <literal>0.0</literal>.
</para>
+
+ <para>
+ The <link linkend="monitoring-pg-stat-autovacuum-priority-view">
+ <structname>pg_stat_autovacuum_priority</structname></link> view can be
+ used to inspect each table's autovacuum eligibility and priority score.
+ </para>
</sect3>
</sect2>
</sect1>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 462019a972c..901dd704804 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -463,6 +463,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</entry>
</row>
+ <row>
+ <entry><structname>pg_stat_autovacuum_priority</structname><indexterm><primary>pg_stat_autovacuum_priority</primary></indexterm></entry>
+ <entry>One row per relation in the current database, showing
+ a table's autovacuum eligibility and priority. See
+ <link linkend="monitoring-pg-stat-autovacuum-priority-view">
+ <structname>pg_stat_autovacuum_priority</structname></link> for details.
+ </entry>
+ </row>
+
<row>
<entry><structname>pg_stat_bgwriter</structname><indexterm><primary>pg_stat_bgwriter</primary></indexterm></entry>
<entry>One row only, showing statistics about the
@@ -2847,6 +2856,105 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para>
</sect2>
+ <sect2 id="monitoring-pg-stat-autovacuum-priority-view">
+ <title><structname>pg_stat_autovacuum_priority</structname></title>
+
+ <indexterm>
+ <primary>pg_stat_autovacuum_priority</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_stat_autovacuum_priority</structname> view contains
+ one row per relation in the current database, showing a table's
+ autovacuum eligibility and priority.
+ </para>
+
+ <table id="pg-stat-autovacuum-priority-view" xreflabel="pg_stat_autovacuum_priority">
+ <title><structname>pg_stat_autovacuum_priority</structname> View</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of the relation
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schemaname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema that this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the relation
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>needs_vacuum</structfield> <type>boolean</type>
+ </para>
+ <para>
+ True if autovacuum considers this relation in need of vacuuming
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>needs_analyze</structfield> <type>boolean</type>
+ </para>
+ <para>
+ True if autovacuum considers this relation in need of analyzing
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound</structfield> <type>boolean</type>
+ </para>
+ <para>
+ True if vacuuming is needed to prevent transaction ID or
+ multixact ID wraparound
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>score</structfield> <type>double precision</type>
+ </para>
+ <para>
+ Priority score used by autovacuum to order which relations to
+ process first. Higher values indicate greater urgency. Zero if
+ the relation does not currently need vacuuming or analyzing.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ </sect2>
+
<sect2 id="monitoring-pg-stat-io-view">
<title><structname>pg_stat_io</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index f1ed7b58f13..f6ec7653204 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -795,6 +795,19 @@ CREATE VIEW pg_stat_xact_user_tables AS
WHERE schemaname NOT IN ('pg_catalog', 'information_schema') AND
schemaname !~ '^pg_toast';
+CREATE VIEW pg_stat_autovacuum_priority AS
+ SELECT
+ S.relid,
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ S.needs_vacuum,
+ S.needs_analyze,
+ S.wraparound,
+ S.score
+ FROM pg_stat_get_autovacuum_priority() S
+ JOIN pg_class C ON C.oid = S.relid
+ LEFT JOIN pg_namespace N ON N.oid = C.relnamespace;
+
CREATE VIEW pg_statio_all_tables AS
SELECT
C.oid AS relid,
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index b5c153a8835..8a73f167653 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -80,6 +80,7 @@
#include "catalog/pg_namespace.h"
#include "commands/vacuum.h"
#include "common/int.h"
+#include "funcapi.h"
#include "lib/ilist.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
@@ -111,6 +112,7 @@
#include "utils/syscache.h"
#include "utils/timeout.h"
#include "utils/timestamp.h"
+#include "utils/tuplestore.h"
#include "utils/wait_event.h"
@@ -372,6 +374,10 @@ static void perform_work_item(AutoVacuumWorkItem *workitem);
static void autovac_report_activity(autovac_table *tab);
static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
const char *nspname, const char *relname);
+static void compute_autovac_score(HeapTuple tuple, TupleDesc pg_class_desc,
+ int effective_multixact_freeze_max_age,
+ bool *dovacuum, bool *doanalyze,
+ bool *wraparound, double *score);
static void avl_sigusr2_handler(SIGNAL_ARGS);
static bool av_worker_available(void);
static void check_av_worker_gucs(void);
@@ -2057,6 +2063,13 @@ do_autovacuum(void)
&dovacuum, &doanalyze, &wraparound,
&score);
+ elog(DEBUG3, "%s: dovacuum: %s, doanalyze: %s, wraparound: %s, score: %.3f",
+ NameStr(classForm->relname),
+ dovacuum ? "yes" : "no",
+ doanalyze ? "yes" : "no",
+ wraparound ? "yes" : "no",
+ score);
+
/* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
{
@@ -2157,6 +2170,12 @@ do_autovacuum(void)
&dovacuum, &doanalyze, &wraparound,
&score);
+ elog(DEBUG3, "%s: dovacuum: %s, wraparound: %s, score: %.3f",
+ NameStr(classForm->relname),
+ dovacuum ? "yes" : "no",
+ wraparound ? "yes" : "no",
+ score);
+
/* ignore analyze for toast tables */
if (dovacuum)
{
@@ -3312,15 +3331,6 @@ relation_needs_vacanalyze(Oid relid,
*score = Max(*score, anlthresh_score);
*doanalyze = true;
}
-
- if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f), score: %.3f",
- NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh, *score);
- else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f), score: %.3f",
- NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh, *score);
}
}
@@ -3635,3 +3645,107 @@ check_av_worker_gucs(void)
errdetail("The server will only start up to \"autovacuum_worker_slots\" (%d) autovacuum workers at a given time.",
autovacuum_worker_slots)));
}
+
+/*
+ * compute_autovac_score
+ * Wrapper around relation_needs_vacanalyze() that handles the
+ * per-relation setup similar to do_autovacuum() before calling
+ * relation_needs_vacanalyze().
+ */
+static void
+compute_autovac_score(HeapTuple tuple, TupleDesc pg_class_desc,
+ int effective_multixact_freeze_max_age,
+ bool *dovacuum, bool *doanalyze,
+ bool *wraparound, double *score)
+{
+ Form_pg_class classForm = (Form_pg_class) GETSTRUCT(tuple);
+ AutoVacOpts *relopts;
+ PgStat_StatTabEntry *tabentry;
+
+ relopts = extract_autovac_opts(tuple, pg_class_desc);
+
+ tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
+ classForm->oid);
+
+ relation_needs_vacanalyze(classForm->oid, relopts, classForm, tabentry,
+ effective_multixact_freeze_max_age,
+ dovacuum, doanalyze, wraparound, score);
+
+ if (relopts)
+ pfree(relopts);
+ if (tabentry)
+ pfree(tabentry);
+}
+
+/*
+ * pg_stat_get_autovacuum_priority
+ * Returns the autovacuum priority score for each relation in the
+ * current database.
+ *
+ * This follows the same setup as do_autovacuum(): snapshotting
+ * recentXid/recentMulti, scanning pg_class, filtering relation kinds
+ * and temp tables, and computing effective_multixact_freeze_max_age
+ * are done here, while compute_autovac_score() handles the per-relation
+ * setup (fetching reloptions and the pgstat entry).
+ */
+#define NUM_AV_SCORE_COLS 5
+
+Datum
+pg_stat_get_autovacuum_priority(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Relation classRel;
+ TableScanDesc relScan;
+ HeapTuple tuple;
+ TupleDesc pg_class_desc;
+ int effective_multixact_freeze_max_age;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ effective_multixact_freeze_max_age = MultiXactMemberFreezeThreshold();
+
+ /* Snapshot once before the scan, like do_autovacuum()'s caller. */
+ recentXid = ReadNextTransactionId();
+ recentMulti = ReadNextMultiXactId();
+
+ classRel = table_open(RelationRelationId, AccessShareLock);
+ pg_class_desc = CreateTupleDescCopy(RelationGetDescr(classRel));
+
+ relScan = table_beginscan_catalog(classRel, 0, NULL);
+ while ((tuple = heap_getnext(relScan, ForwardScanDirection)) != NULL)
+ {
+ Form_pg_class classForm = (Form_pg_class) GETSTRUCT(tuple);
+ bool dovacuum;
+ bool doanalyze;
+ bool wraparound;
+ double score = 0.0;
+ Datum values[NUM_AV_SCORE_COLS];
+ bool nulls[NUM_AV_SCORE_COLS] = {false};
+
+ if (classForm->relkind != RELKIND_RELATION &&
+ classForm->relkind != RELKIND_MATVIEW &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ continue;
+
+ if (classForm->relpersistence == RELPERSISTENCE_TEMP)
+ continue;
+
+ compute_autovac_score(tuple, pg_class_desc,
+ effective_multixact_freeze_max_age,
+ &dovacuum, &doanalyze, &wraparound, &score);
+
+ values[0] = ObjectIdGetDatum(classForm->oid);
+ values[1] = BoolGetDatum(dovacuum);
+ values[2] = BoolGetDatum(doanalyze);
+ values[3] = BoolGetDatum(wraparound);
+ values[4] = Float8GetDatum(score);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ table_endscan(relScan);
+
+ table_close(classRel, AccessShareLock);
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index 420850293f8..bce64758823 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202603201
+#define CATALOG_VERSION_NO 202603231
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 84e7adde0e5..e52420898ce 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5667,6 +5667,15 @@
proname => 'pg_stat_get_total_autoanalyze_time', provolatile => 's',
proparallel => 'r', prorettype => 'float8', proargtypes => 'oid',
prosrc => 'pg_stat_get_total_autoanalyze_time' },
+{ oid => '8409',
+ descr => 'statistics: autovacuum priority scores for all relations',
+ proname => 'pg_stat_get_autovacuum_priority', prorows => '100',
+ proretset => 't', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => '',
+ proallargtypes => '{oid,bool,bool,bool,float8}',
+ proargmodes => '{o,o,o,o,o}',
+ proargnames => '{relid,needs_vacuum,needs_analyze,wraparound,score}',
+ prosrc => 'pg_stat_get_autovacuum_priority' },
{ oid => '1936', descr => 'statistics: currently active backend IDs',
proname => 'pg_stat_get_backend_idset', prorows => '100', proretset => 't',
provolatile => 's', proparallel => 'r', prorettype => 'int4',
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 32bea58db2c..257f21be004 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1860,6 +1860,16 @@ pg_stat_archiver| SELECT archived_count,
last_failed_time,
stats_reset
FROM pg_stat_get_archiver() s(archived_count, last_archived_wal, last_archived_time, failed_count, last_failed_wal, last_failed_time, stats_reset);
+pg_stat_autovacuum_priority| SELECT s.relid,
+ n.nspname AS schemaname,
+ c.relname,
+ s.needs_vacuum,
+ s.needs_analyze,
+ s.wraparound,
+ s.score
+ FROM ((pg_stat_get_autovacuum_priority() s(relid, needs_vacuum, needs_analyze, wraparound, score)
+ JOIN pg_class c ON ((c.oid = s.relid)))
+ LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)));
pg_stat_bgwriter| SELECT pg_stat_get_bgwriter_buf_written_clean() AS buffers_clean,
pg_stat_get_bgwriter_maxwritten_clean() AS maxwritten_clean,
pg_stat_get_buf_alloc() AS buffers_alloc,
--
2.47.3
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2026-03-23 21:01 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-23 21:01 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Mon, Mar 23, 2026 at 02:01:22PM -0500, Sami Imseih wrote:
> Adding a system view will be nice. I am attaching a version I used in earlier
> testing (cleaned up with docs), if we are inclined to get this in. I
> think it will be
> useful.
Thanks. IMHO we should continue to focus on the main patch and get that
committed first.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-23 22:27 ` Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Jim Nasby @ 2026-03-23 22:27 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Mon, Mar 23, 2026 at 4:01 PM Nathan Bossart <nathandbossart@gmail.com>
wrote:
> On Mon, Mar 23, 2026 at 02:01:22PM -0500, Sami Imseih wrote:
> > Adding a system view will be nice. I am attaching a version I used in
> earlier
> > testing (cleaned up with docs), if we are inclined to get this in. I
> > think it will be
> > useful.
>
> Thanks. IMHO we should continue to focus on the main patch and get that
> committed first.
>
+1 ... for one thing if we're going to add a view meant for monitoring
autovac decisions I'd like to think about ways to measure how many tables
are "close" to being eligible for autovac. In particular, the scenario
where you've just done an MVU via some form of logical, so now the freeze
ages on all your tables are extremely similar.
It might be nice if we had an official means of publishing things that
we'd really like users to kick the tires on, but hold a clear understanding
that we promise no backwards compatibility, promise we'll keep supporting,
etc. I can't see how that'd work with backend code, but could certainly be
done for anything in userspace.
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
@ 2026-03-24 00:32 ` David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2026-03-24 00:32 UTC (permalink / raw)
To: Jim Nasby <jnasby@upgrade.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Sami Imseih <samimseih@gmail.com>; Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, 24 Mar 2026 at 11:27, Jim Nasby <jnasby@upgrade.com> wrote:
>
> On Mon, Mar 23, 2026 at 4:01 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
>> Thanks. IMHO we should continue to focus on the main patch and get that
>> committed first.
>
>
> +1 ... for one thing if we're going to add a view meant for monitoring autovac decisions I'd like to think about ways to measure how many tables are "close" to being eligible for autovac. In particular, the scenario where you've just done an MVU via some form of logical, so now the freeze ages on all your tables are extremely similar.
+1 for main patch first. I do think a view would be useful as a
follow-up. However, which columns we put in that view might have some
influence on how the current patch should look. I think the view
should show the individual scores and the total score as the Max() of
the individual scores. If we didn't do that, it might be confusing to
the user which aspect of the score the final score is derived from.
That might mean that it'd be better to have
relation_needs_vacanalyze() output the scores individually, or perhaps
populate a struct that we pass in that gets allocated on the stack
during do_autovacuum(). That'd mean a bit less churn if we go with the
view containing individual scores.
I think it would be good to have the view show tables that are not
eligible for autovacuum too. It should be easy for users to filter
those out for cases where they're not needed. Doing that would make it
very easy for anyone who wanted to code up a script to run off-peak to
vacuum tables that might need attention on the next peak. Something
like:
SELECT 'VACUUM ' || vacrelid::regclass || ';' from
pg_stat_autovacuum_priority WHERE vacuum_score BETWEEN 0.75 AND 1.0
ORDER BY vacuum_score DESC;
\gexec
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2026-03-24 17:15 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-24 21:12 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Nathan Bossart @ 2026-03-24 17:15 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Mar 24, 2026 at 01:32:40PM +1300, David Rowley wrote:
> +1 for main patch first. I do think a view would be useful as a
> follow-up. However, which columns we put in that view might have some
> influence on how the current patch should look. I think the view
> should show the individual scores and the total score as the Max() of
> the individual scores. If we didn't do that, it might be confusing to
> the user which aspect of the score the final score is derived from.
> That might mean that it'd be better to have
> relation_needs_vacanalyze() output the scores individually, or perhaps
> populate a struct that we pass in that gets allocated on the stack
> during do_autovacuum(). That'd mean a bit less churn if we go with the
> view containing individual scores.
Agreed. Here's a first try at that. I also updated the DEBUG3 at the end
of relation_needs_vacanalyze() to show the individual scores. The comment
above that function might need some work, and we might need a bit of
additional commentary elsewhere.
--
nathan
From f2f0b6a1f3b7593dea9a4800735fa21f9b8371e0 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v15 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 ++++++
doc/src/sgml/maintenance.sgml | 92 ++++++
src/backend/postmaster/autovacuum.c | 266 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 453 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 75c22405a09..b5a191c130b 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1062,6 +1062,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 219673db930..35b9206a51a 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -312,6 +317,22 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ double max;
+ double xid;
+ double mxid;
+ double vac;
+ double vac_ins;
+ double anl;
+} AutoVacuumScores;
+
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +371,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ AutoVacuumScores *scores);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1867,6 +1889,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1880,7 +1911,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1992,6 +2023,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2032,11 +2064,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2092,6 +2131,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2124,11 +2164,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2252,6 +2299,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2280,9 +2329,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2513,7 +2562,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2915,6 +2964,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ AutoVacuumScores scores;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2922,15 +2972,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &scores);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2971,6 +3018,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 10.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2981,7 +3065,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ AutoVacuumScores *scores)
{
bool force_vacuum;
bool av_enabled;
@@ -3010,11 +3095,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ memset(scores, 0, sizeof(AutoVacuumScores));
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3062,17 +3152,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3081,13 +3171,65 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ scores->xid = (double) xid_age / freeze_max_age;
+ scores->mxid = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * begin scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ scores->xid = pow(scores->xid, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ scores->mxid = pow(scores->mxid, Max(1.0, (double) mxid_age / 100000000));
+
+ scores->xid *= autovacuum_freeze_score_weight;
+ scores->mxid *= autovacuum_multixact_freeze_score_weight;
+
+ scores->max = Max(scores->xid, scores->mxid);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3136,34 +3278,64 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ scores->vac = (double) vactuples / Max(vacthresh, 1);
+ scores->vac *= autovacuum_vacuum_score_weight;
+ scores->max = Max(scores->max, scores->vac);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ scores->vac_ins = (double) instuples / Max(vacinsthresh, 1);
+ scores->vac_ins *= autovacuum_vacuum_insert_score_weight;
+ scores->max = Max(scores->max, scores->vac_ins);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ scores->anl = (double) anltuples / Max(anlthresh, 1);
+ scores->anl *= autovacuum_analyze_score_weight;
+ scores->max = Max(scores->max, scores->anl);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3,
+ "%s: "
+ "vac: %.0f (thresh %.0f, score %.2f), "
+ "ins: %.0f (thresh %.0f, score %.2f), "
+ "anl: %.0f (thresh %.0f, score %.2f), "
+ "xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, scores->vac,
+ instuples, vacinsthresh, scores->vac_ins,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3,
+ "%s: "
+ "vac: %.0f (thresh %.0f, score %.2f), "
+ "ins: (disabled), "
+ "anl: %.0f (thresh %.0f, score %.2f), "
+ "xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, scores->vac,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8df23840e57..43ae8ddffc8 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -187,6 +187,7 @@ AuthToken
AutoPrewarmReadStreamData
AutoPrewarmSharedState
AutoVacOpts
+AutoVacuumScores
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
@@ -3098,6 +3099,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v15-0001-autovacuum-scheduling-improvements.patch (30.4K, ../../acLGoktU2lp4SIlU@nathan/2-v15-0001-autovacuum-scheduling-improvements.patch)
download | inline diff:
From f2f0b6a1f3b7593dea9a4800735fa21f9b8371e0 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 10 Oct 2025 12:28:37 -0500
Subject: [PATCH v15 1/1] autovacuum scheduling improvements
---
doc/src/sgml/config.sgml | 90 ++++++
doc/src/sgml/maintenance.sgml | 92 ++++++
src/backend/postmaster/autovacuum.c | 266 ++++++++++++++----
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 453 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 75c22405a09..b5a191c130b 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1062,6 +1062,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 219673db930..35b9206a51a 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -312,6 +317,22 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+typedef struct
+{
+ double max;
+ double xid;
+ double mxid;
+ double vac;
+ double vac_ins;
+ double anl;
+} AutoVacuumScores;
+
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -350,7 +371,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ AutoVacuumScores *scores);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1867,6 +1889,15 @@ get_database_list(void)
return dblist;
}
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1880,7 +1911,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1992,6 +2023,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2032,11 +2064,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2092,6 +2131,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2124,11 +2164,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2252,6 +2299,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2280,9 +2329,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2513,7 +2562,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2915,6 +2964,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ AutoVacuumScores scores;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2922,15 +2972,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &scores);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2971,6 +3018,43 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns a score that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum gets to processing it. However, we have attempted to
+ * develop something that is expected to work for a large portion of workloads
+ * with reasonable parameter settings.
+ *
+ * The score is calculated as the maximum of the ratios of each of the table's
+ * relevant values to its threshold. For example, if the number of inserted
+ * tuples is 100, and the insert threshold for the table is 80, the insert
+ * score is 1.25. If all other scores are below that value, the returned score
+ * will be 1.25. The other criteria considered for the score are the table
+ * ages (both relfrozenxid and relminmxid) compared to the corresponding
+ * freeze-max-age setting, the number of updated/deleted tuples compared to the
+ * vacuum threshold, and the number of inserted/updated/deleted tuples compared
+ * to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the top of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 10.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2981,7 +3065,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ AutoVacuumScores *scores)
{
bool force_vacuum;
bool av_enabled;
@@ -3010,11 +3095,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ memset(scores, 0, sizeof(AutoVacuumScores));
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3062,17 +3152,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3081,13 +3171,65 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ scores->xid = (double) xid_age / freeze_max_age;
+ scores->mxid = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * begin scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ scores->xid = pow(scores->xid, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ scores->mxid = pow(scores->mxid, Max(1.0, (double) mxid_age / 100000000));
+
+ scores->xid *= autovacuum_freeze_score_weight;
+ scores->mxid *= autovacuum_multixact_freeze_score_weight;
+
+ scores->max = Max(scores->xid, scores->mxid);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3136,34 +3278,64 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ scores->vac = (double) vactuples / Max(vacthresh, 1);
+ scores->vac *= autovacuum_vacuum_score_weight;
+ scores->max = Max(scores->max, scores->vac);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ scores->vac_ins = (double) instuples / Max(vacinsthresh, 1);
+ scores->vac_ins *= autovacuum_vacuum_insert_score_weight;
+ scores->max = Max(scores->max, scores->vac_ins);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ scores->anl = (double) anltuples / Max(anlthresh, 1);
+ scores->anl *= autovacuum_analyze_score_weight;
+ scores->max = Max(scores->max, scores->anl);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3,
+ "%s: "
+ "vac: %.0f (thresh %.0f, score %.2f), "
+ "ins: %.0f (thresh %.0f, score %.2f), "
+ "anl: %.0f (thresh %.0f, score %.2f), "
+ "xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, scores->vac,
+ instuples, vacinsthresh, scores->vac_ins,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3,
+ "%s: "
+ "vac: %.0f (thresh %.0f, score %.2f), "
+ "ins: (disabled), "
+ "anl: %.0f (thresh %.0f, score %.2f), "
+ "xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
- }
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
+ vactuples, vacthresh, scores->vac,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
}
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8df23840e57..43ae8ddffc8 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -187,6 +187,7 @@ AuthToken
AutoPrewarmReadStreamData
AutoPrewarmSharedState
AutoVacOpts
+AutoVacuumScores
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
@@ -3098,6 +3099,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-24 21:12 ` Sami Imseih <samimseih@gmail.com>
1 sibling, 0 replies; 150+ messages in thread
From: Sami Imseih @ 2026-03-24 21:12 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> On Tue, Mar 24, 2026 at 01:32:40PM +1300, David Rowley wrote:
> > +1 for main patch first. I do think a view would be useful as a
> > follow-up. However, which columns we put in that view might have some
> > influence on how the current patch should look. I think the view
> > should show the individual scores and the total score as the Max() of
> > the individual scores. If we didn't do that, it might be confusing to
> > the user which aspect of the score the final score is derived from.
> > That might mean that it'd be better to have
> > relation_needs_vacanalyze() output the scores individually, or perhaps
> > populate a struct that we pass in that gets allocated on the stack
> > during do_autovacuum(). That'd mean a bit less churn if we go with the
> > view containing individual scores.
>
> Agreed. Here's a first try at that. I also updated the DEBUG3 at the end
> of relation_needs_vacanalyze() to show the individual scores. The comment
> above that function might need some work, and we might need a bit of
> additional commentary elsewhere.
This is good and these values could be exposed in the future view
individually. I like this.
v15 also LGTM.
--
Sami
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-25 21:12 ` Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Bharath Rupireddy @ 2026-03-25 21:12 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Hi,
On Tue, Mar 24, 2026 at 10:15 AM Nathan Bossart
<nathandbossart@gmail.com> wrote:
>
> Agreed. Here's a first try at that. I also updated the DEBUG3 at the end
> of relation_needs_vacanalyze() to show the individual scores. The comment
> above that function might need some work, and we might need a bit of
> additional commentary elsewhere.
Sorry for the late response. Thank you for sending the latest patch.
+1 for getting the main patch in first and then the scoring stats view.
I looked closely at the v15 patch today and it LGTM.
A couple of thoughts (I don't intend to block the main patch from
getting in :)):
1/ If a large bloated table scores highest and takes hours to vacuum,
other tables' XID ages keep advancing (not to the failsafe limits yet,
but approaching close to them). By the time the autovacuum worker
finishes, a table that now needs freezing is still stuck at its old
position in the sorted table list.
Would it make sense to recompute scores and re-sort the remaining
table list after each table is processed in do_autovacuum()'s main
loop - say, after a certain amount of time spent vacuuming the large
table(s)? This would catch the above scenarios. I see that the scores
per table are being calculated in relation_needs_vacanalyze, but they
are ignored in the recheck path (table_recheck_autovac ->
recheck_relation_needs_vacanalyze -> relation_needs_vacanalyze).
IMHO, this could be a useful future addition.
2/ Nit: IIUC, autovacuum_vacuum_score_weight and other existing vacuum
thresholds and scale factor GUCs are the ones to tune to make the
tables prioritized first. I still think it would be nice to have a
document explaining these scenarios - maybe after the main patch gets
in.
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
@ 2026-03-25 21:18 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:24 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 2 replies; 150+ messages in thread
From: Nathan Bossart @ 2026-03-25 21:18 UTC (permalink / raw)
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, Mar 25, 2026 at 02:12:16PM -0700, Bharath Rupireddy wrote:
> Would it make sense to recompute scores and re-sort the remaining
> table list after each table is processed in do_autovacuum()'s main
> loop - say, after a certain amount of time spent vacuuming the large
> table(s)? This would catch the above scenarios. I see that the scores
> per table are being calculated in relation_needs_vacanalyze, but they
> are ignored in the recheck path (table_recheck_autovac ->
> recheck_relation_needs_vacanalyze -> relation_needs_vacanalyze).
I think this was discussed a bit upthread, and we decided to leave it out
for now. But things like reprioritization and automatic cost limit
adjustments seem worth considering for v20.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-25 21:24 ` Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
1 sibling, 0 replies; 150+ messages in thread
From: Bharath Rupireddy @ 2026-03-25 21:24 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Hi,
On Wed, Mar 25, 2026 at 2:18 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> On Wed, Mar 25, 2026 at 02:12:16PM -0700, Bharath Rupireddy wrote:
> > Would it make sense to recompute scores and re-sort the remaining
> > table list after each table is processed in do_autovacuum()'s main
> > loop - say, after a certain amount of time spent vacuuming the large
> > table(s)? This would catch the above scenarios. I see that the scores
> > per table are being calculated in relation_needs_vacanalyze, but they
> > are ignored in the recheck path (table_recheck_autovac ->
> > recheck_relation_needs_vacanalyze -> relation_needs_vacanalyze).
>
> I think this was discussed a bit upthread, and we decided to leave it out
> for now. But things like reprioritization and automatic cost limit
> adjustments seem worth considering for v20.
+1. Thanks for the clarification.
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-25 22:00 ` David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: David Rowley @ 2026-03-25 22:00 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, 26 Mar 2026 at 10:18, Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> On Wed, Mar 25, 2026 at 02:12:16PM -0700, Bharath Rupireddy wrote:
> > Would it make sense to recompute scores and re-sort the remaining
> > table list after each table is processed in do_autovacuum()'s main
> > loop - say, after a certain amount of time spent vacuuming the large
> > table(s)? This would catch the above scenarios. I see that the scores
> > per table are being calculated in relation_needs_vacanalyze, but they
> > are ignored in the recheck path (table_recheck_autovac ->
> > recheck_relation_needs_vacanalyze -> relation_needs_vacanalyze).
>
> I think this was discussed a bit upthread, and we decided to leave it out
> for now. But things like reprioritization and automatic cost limit
> adjustments seem worth considering for v20.
Agreed. I think the reason you mentioned in [1] was a good reason not
to do this.
There are also other autovacuum workers that may be calculating a more
up-to-date list. They may well process the table that's increased
score before the worker with the slightly stale list makes it there.
That seems fine and natural to me.
David
[1] https://postgr.es/m/aROY-MUVO_mYTl2f%40nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2026-03-25 22:10 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-25 22:10 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Here is what I have staged for commit. I'll give this at least one more
close read-through beforehand, but I'm hoping to commit it Thursday or
Friday. Thanks everybody for the thoughtful discussion.
--
nathan
From 8c5b3608e583f6be92df054ec8ac7c278c28c7b9 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 25 Mar 2026 16:07:11 -0500
Subject: [PATCH v16 1/1] Add rudimentary table prioritization to autovacuum.
Autovacuum workers scan pg_class twice to collect the set of tables
to process. The first pass is for plain relations and materialized
views, and the second is for TOAST tables. When the worker finds a
table to process, it adds it to the end of a list. Later on, it
processes the tables in the same order as the list. This simple
strategy has worked surprisingly well for a long time, but there
have been many discussions over the years about trying to make it
better.
This commit introduces a scoring system that is used to sort the
aforementioned list of tables to process. The idea is to have
autovacuum workers prioritize tables that are furthest beyond their
thresholds (e.g., a table nearing transaction ID wraparound should
be vacuumed first). This prioritization scheme is certainly far
from perfect; there are simply too many possibilities for any
scoring technique to work across all workloads, and the situation
might change significantly between the time we calculate the score
and the time that autovacuum processes it. However, we have
attemped to develop something that is expected to work for a large
portion of workloads with reasonable parameter settings.
The score is calculated as the maximum of the ratios of each of the
table's relevant values to its threshold. For example, if the
number of inserted tuples is 100, and the insert threshold for the
table is 80, the insert score is 1.25. If all other scores are
below that value, the table's score will be 1.25. The other
criteria considered for the score are the table ages (both
relfrozenxid and relminmxid) compared to the corresponding
freeze-max-age setting, the number of update/deleted tuples
compared to the vacuum threshold, and the number of
inserted/updated/deleted tuples compared to the analyze threshold.
Once exception to the previous paragraph is for tables nearing
wraparound, i.e., those that have surpassed the effective failsafe
ages. In that case, the relfrozenxid/relminmxid-based score is
scaled aggressively so that the table has a decent chance of
sorting to the front of the list.
To adjust how strongly each component contributes to the score, the
following parameters can be adjusted from their default of 1.0 to
anywhere between 0.0 and 10.0 (inclusive). Setting all of these to
0.0 restores pre-v19 prioritization behavior:
autovacuum_freeze_score_weight
autovacuum_multixact_freeze_score_weight
autovacuum_vacuum_score_weight
autovacuum_vacuum_insert_score_weight
autovacuum_analyze_score_weight
This is intended to be a baby step towards smarter autovacuum
workers. Possible future improvements include, but are not limited
to, periodic reprioritization, automatic cost limit adjustments,
and better observability (e.g., a system view that shows current
scores). While we do not expect this commit to produce
earth-shattering improvements, it should in theory produce marginal
ones, and it is arguably a prerequisite for the aforementioned
follow-up changes.
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Reviewed-by: Greg Burd <greg@burd.me>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/aOaAuXREwnPZVISO%40nathan
---
doc/src/sgml/config.sgml | 90 ++++++
doc/src/sgml/maintenance.sgml | 92 ++++++
src/backend/postmaster/autovacuum.c | 282 +++++++++++++++---
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 469 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 75c22405a09..b5a191c130b 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1062,6 +1062,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7ecb069c248..23e3b6f1d19 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +316,30 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+/*
+ * This struct is used by relation_needs_vacanalyze() to return the table's
+ * score (i.e., the maximum of the component scores) as well as the component
+ * scores themselves.
+ */
+typedef struct
+{
+ double max; /* maximum of all values below */
+ double xid; /* transaction ID component */
+ double mxid; /* multixact ID component */
+ double vac; /* vacuum component */
+ double vac_ins; /* vacuum insert component */
+ double anl; /* analyze component */
+} AutoVacuumScores;
+
+/*
+ * This struct is used to track and sort the list of tables to process.
+ */
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +378,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ AutoVacuumScores *scores);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1896,19 @@ get_database_list(void)
return dblist;
}
+/*
+ * List comparator for TableToProcess. Note that this sorts the tables based
+ * on their scores in descending order.
+ */
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1922,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2034,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2075,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2142,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2175,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2310,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2340,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2573,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2913,6 +2974,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ AutoVacuumScores scores;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2920,15 +2982,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &scores);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2969,6 +3028,47 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns scores that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum processes it. However, we have attempted to develop
+ * something that is expected to work for a large portion of workloads with
+ * reasonable parameter settings.
+ *
+ * The autovacuum table score is calculated as the maximum of the ratios of
+ * each of the table's relevant values to its threshold. For example, if the
+ * number of inserted tuples is 100, and the insert threshold for the table is
+ * 80, the insert score is 1.25. If all other scores are below that value, the
+ * returned score will be 1.25. The other criteria considered for the score
+ * are the table ages (both relfrozenxid and relminmxid) compared to the
+ * corresponding freeze-max-age setting, the number of updated/deleted tuples
+ * compared to the vacuum threshold, and the number of inserted/updated/deleted
+ * tuples compared to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the front of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 10.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
+ *
+ * The autovacuum table score is returned in scores->max. The component scores
+ * are also returned in the "scores" argument via the other members of the
+ * AutoVacuumScores struct.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2979,7 +3079,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ AutoVacuumScores *scores)
{
bool force_vacuum;
bool av_enabled;
@@ -3008,11 +3109,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ memset(scores, 0, sizeof(AutoVacuumScores));
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3060,17 +3166,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3079,13 +3185,65 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ scores->xid = (double) xid_age / freeze_max_age;
+ scores->mxid = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * begin scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ scores->xid = pow(scores->xid, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ scores->mxid = pow(scores->mxid, Max(1.0, (double) mxid_age / 100000000));
+
+ scores->xid *= autovacuum_freeze_score_weight;
+ scores->mxid *= autovacuum_multixact_freeze_score_weight;
+
+ scores->max = Max(scores->xid, scores->mxid);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3134,34 +3292,64 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ scores->vac = (double) vactuples / Max(vacthresh, 1);
+ scores->vac *= autovacuum_vacuum_score_weight;
+ scores->max = Max(scores->max, scores->vac);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ scores->vac_ins = (double) instuples / Max(vacinsthresh, 1);
+ scores->vac_ins *= autovacuum_vacuum_insert_score_weight;
+ scores->max = Max(scores->max, scores->vac_ins);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ scores->anl = (double) anltuples / Max(anlthresh, 1);
+ scores->anl *= autovacuum_analyze_score_weight;
+ scores->max = Max(scores->max, scores->anl);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3,
+ "%s: "
+ "vac: %.0f (thresh %.0f, score %.2f), "
+ "ins: %.0f (thresh %.0f, score %.2f), "
+ "anl: %.0f (thresh %.0f, score %.2f), "
+ "xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, scores->vac,
+ instuples, vacinsthresh, scores->vac_ins,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3,
+ "%s: "
+ "vac: %.0f (thresh %.0f, score %.2f), "
+ "ins: (disabled), "
+ "anl: %.0f (thresh %.0f, score %.2f), "
+ "xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ vactuples, vacthresh, scores->vac,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
}
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index dbbec84b222..eb781e74c43 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -187,6 +187,7 @@ AuthToken
AutoPrewarmReadStreamData
AutoPrewarmSharedState
AutoVacOpts
+AutoVacuumScores
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
@@ -3098,6 +3099,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v16-0001-Add-rudimentary-table-prioritization-to-autovacu.patch (34.5K, ../../acRdY_cxDvff5zW4@nathan/2-v16-0001-Add-rudimentary-table-prioritization-to-autovacu.patch)
download | inline diff:
From 8c5b3608e583f6be92df054ec8ac7c278c28c7b9 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 25 Mar 2026 16:07:11 -0500
Subject: [PATCH v16 1/1] Add rudimentary table prioritization to autovacuum.
Autovacuum workers scan pg_class twice to collect the set of tables
to process. The first pass is for plain relations and materialized
views, and the second is for TOAST tables. When the worker finds a
table to process, it adds it to the end of a list. Later on, it
processes the tables in the same order as the list. This simple
strategy has worked surprisingly well for a long time, but there
have been many discussions over the years about trying to make it
better.
This commit introduces a scoring system that is used to sort the
aforementioned list of tables to process. The idea is to have
autovacuum workers prioritize tables that are furthest beyond their
thresholds (e.g., a table nearing transaction ID wraparound should
be vacuumed first). This prioritization scheme is certainly far
from perfect; there are simply too many possibilities for any
scoring technique to work across all workloads, and the situation
might change significantly between the time we calculate the score
and the time that autovacuum processes it. However, we have
attemped to develop something that is expected to work for a large
portion of workloads with reasonable parameter settings.
The score is calculated as the maximum of the ratios of each of the
table's relevant values to its threshold. For example, if the
number of inserted tuples is 100, and the insert threshold for the
table is 80, the insert score is 1.25. If all other scores are
below that value, the table's score will be 1.25. The other
criteria considered for the score are the table ages (both
relfrozenxid and relminmxid) compared to the corresponding
freeze-max-age setting, the number of update/deleted tuples
compared to the vacuum threshold, and the number of
inserted/updated/deleted tuples compared to the analyze threshold.
Once exception to the previous paragraph is for tables nearing
wraparound, i.e., those that have surpassed the effective failsafe
ages. In that case, the relfrozenxid/relminmxid-based score is
scaled aggressively so that the table has a decent chance of
sorting to the front of the list.
To adjust how strongly each component contributes to the score, the
following parameters can be adjusted from their default of 1.0 to
anywhere between 0.0 and 10.0 (inclusive). Setting all of these to
0.0 restores pre-v19 prioritization behavior:
autovacuum_freeze_score_weight
autovacuum_multixact_freeze_score_weight
autovacuum_vacuum_score_weight
autovacuum_vacuum_insert_score_weight
autovacuum_analyze_score_weight
This is intended to be a baby step towards smarter autovacuum
workers. Possible future improvements include, but are not limited
to, periodic reprioritization, automatic cost limit adjustments,
and better observability (e.g., a system view that shows current
scores). While we do not expect this commit to produce
earth-shattering improvements, it should in theory produce marginal
ones, and it is arguably a prerequisite for the aforementioned
follow-up changes.
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Reviewed-by: Greg Burd <greg@burd.me>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/aOaAuXREwnPZVISO%40nathan
---
doc/src/sgml/config.sgml | 90 ++++++
doc/src/sgml/maintenance.sgml | 92 ++++++
src/backend/postmaster/autovacuum.c | 282 +++++++++++++++---
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 469 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 75c22405a09..b5a191c130b 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1062,6 +1062,98 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7ecb069c248..23e3b6f1d19 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +316,30 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+/*
+ * This struct is used by relation_needs_vacanalyze() to return the table's
+ * score (i.e., the maximum of the component scores) as well as the component
+ * scores themselves.
+ */
+typedef struct
+{
+ double max; /* maximum of all values below */
+ double xid; /* transaction ID component */
+ double mxid; /* multixact ID component */
+ double vac; /* vacuum component */
+ double vac_ins; /* vacuum insert component */
+ double anl; /* analyze component */
+} AutoVacuumScores;
+
+/*
+ * This struct is used to track and sort the list of tables to process.
+ */
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +378,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ AutoVacuumScores *scores);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1896,19 @@ get_database_list(void)
return dblist;
}
+/*
+ * List comparator for TableToProcess. Note that this sorts the tables based
+ * on their scores in descending order.
+ */
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1922,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2034,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2075,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2142,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2175,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2310,8 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2340,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2573,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2913,6 +2974,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ AutoVacuumScores scores;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2920,15 +2982,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &scores);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2969,6 +3028,47 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns scores that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum processes it. However, we have attempted to develop
+ * something that is expected to work for a large portion of workloads with
+ * reasonable parameter settings.
+ *
+ * The autovacuum table score is calculated as the maximum of the ratios of
+ * each of the table's relevant values to its threshold. For example, if the
+ * number of inserted tuples is 100, and the insert threshold for the table is
+ * 80, the insert score is 1.25. If all other scores are below that value, the
+ * returned score will be 1.25. The other criteria considered for the score
+ * are the table ages (both relfrozenxid and relminmxid) compared to the
+ * corresponding freeze-max-age setting, the number of updated/deleted tuples
+ * compared to the vacuum threshold, and the number of inserted/updated/deleted
+ * tuples compared to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the front of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 10.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
+ *
+ * The autovacuum table score is returned in scores->max. The component scores
+ * are also returned in the "scores" argument via the other members of the
+ * AutoVacuumScores struct.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2979,7 +3079,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ AutoVacuumScores *scores)
{
bool force_vacuum;
bool av_enabled;
@@ -3008,11 +3109,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ memset(scores, 0, sizeof(AutoVacuumScores));
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3060,17 +3166,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3079,13 +3185,65 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ scores->xid = (double) xid_age / freeze_max_age;
+ scores->mxid = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * begin scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ scores->xid = pow(scores->xid, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ scores->mxid = pow(scores->mxid, Max(1.0, (double) mxid_age / 100000000));
+
+ scores->xid *= autovacuum_freeze_score_weight;
+ scores->mxid *= autovacuum_multixact_freeze_score_weight;
+
+ scores->max = Max(scores->xid, scores->mxid);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3134,34 +3292,64 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ scores->vac = (double) vactuples / Max(vacthresh, 1);
+ scores->vac *= autovacuum_vacuum_score_weight;
+ scores->max = Max(scores->max, scores->vac);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ scores->vac_ins = (double) instuples / Max(vacinsthresh, 1);
+ scores->vac_ins *= autovacuum_vacuum_insert_score_weight;
+ scores->max = Max(scores->max, scores->vac_ins);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ scores->anl = (double) anltuples / Max(anlthresh, 1);
+ scores->anl *= autovacuum_analyze_score_weight;
+ scores->max = Max(scores->max, scores->anl);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3,
+ "%s: "
+ "vac: %.0f (thresh %.0f, score %.2f), "
+ "ins: %.0f (thresh %.0f, score %.2f), "
+ "anl: %.0f (thresh %.0f, score %.2f), "
+ "xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, scores->vac,
+ instuples, vacinsthresh, scores->vac_ins,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3,
+ "%s: "
+ "vac: %.0f (thresh %.0f, score %.2f), "
+ "ins: (disabled), "
+ "anl: %.0f (thresh %.0f, score %.2f), "
+ "xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ vactuples, vacthresh, scores->vac,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
}
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..ab9cbebbc3f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index dbbec84b222..eb781e74c43 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -187,6 +187,7 @@ AuthToken
AutoPrewarmReadStreamData
AutoPrewarmSharedState
AutoVacOpts
+AutoVacuumScores
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
@@ -3098,6 +3099,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-26 00:28 ` David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2026-03-26 00:28 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, 26 Mar 2026 at 11:10, Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> Here is what I have staged for commit. I'll give this at least one more
> close read-through beforehand, but I'm hoping to commit it Thursday or
> Friday. Thanks everybody for the thoughtful discussion.
A review:
1. I don't think the following is exactly true:
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
Should it mention dividing by the autovacuum_freeze_score_weight?
2. Is it worth expanding the following paragraph with some examples?
For example, raising the priority of analyze, someone might want to
set autovacuum_analyze_score_weight to 2.0, effectively doubling the
analyze scores.
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>.
+ </para>
3. We talked about this a bit, but after reading the below comment and
looking at sort_template.h, I wonder if we might be putting too much
faith into the current qsort implementation. The presorted check
should pass when all the scores are 0.0. There's also the code that
runs before that for when n < 7 which shouldn't do any swapping. I'm
wondering if that comment might be putting a little too much faith
into that remaining true in the future. You're also calling the same
thing out in #2, so maybe we better take some safer measures to ensure
it remains true instead of relying on the current qsort code not
changing.
+ * between 0.0 and 10.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
4. I think we tend to favour not breaking ERROR strings up like this:
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold
%.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3,
+ "%s: "
+ "vac: %.0f (thresh %.0f, score %.2f), "
+ "ins: %.0f (thresh %.0f, score %.2f), "
+ "anl: %.0f (thresh %.0f, score %.2f), "
+ "xid score: %.2f, mxid score: %.2f",
It reduces grepability of error messages. I guess you could argue that
this has so many format specifiers that it's not that greppable
anyway... I sometimes do guess those to grep for things.
5. Maybe worth specifying the range in a comment in the following:
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change
requires restart)
+#autovacuum_freeze_score_weight = 1.0
+#autovacuum_multixact_freeze_score_weight = 1.0
+#autovacuum_vacuum_score_weight = 1.0
+#autovacuum_vacuum_insert_score_weight = 1.0
+#autovacuum_analyze_score_weight = 1.0
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2026-03-26 16:49 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-26 16:49 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Mar 26, 2026 at 01:28:16PM +1300, David Rowley wrote:
> A review:
Thanks. I believe I've addressed all your feedback.
--
nathan
From bf5d0ffc742ce744deb18ed91b8ce844b013eed5 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 25 Mar 2026 16:07:11 -0500
Subject: [PATCH v17 1/1] Add rudimentary table prioritization to autovacuum.
Autovacuum workers scan pg_class twice to collect the set of tables
to process. The first pass is for plain relations and materialized
views, and the second is for TOAST tables. When the worker finds a
table to process, it adds it to the end of a list. Later on, it
processes the tables in the same order as the list. This simple
strategy has worked surprisingly well for a long time, but there
have been many discussions over the years about trying to make it
better.
This commit introduces a scoring system that is used to sort the
aforementioned list of tables to process. The idea is to have
autovacuum workers prioritize tables that are furthest beyond their
thresholds (e.g., a table nearing transaction ID wraparound should
be vacuumed first). This prioritization scheme is certainly far
from perfect; there are simply too many possibilities for any
scoring technique to work across all workloads, and the situation
might change significantly between the time we calculate the score
and the time that autovacuum processes it. However, we have
attemped to develop something that is expected to work for a large
portion of workloads with reasonable parameter settings.
The score is calculated as the maximum of the ratios of each of the
table's relevant values to its threshold. For example, if the
number of inserted tuples is 100, and the insert threshold for the
table is 80, the insert score is 1.25. If all other scores are
below that value, the table's score will be 1.25. The other
criteria considered for the score are the table ages (both
relfrozenxid and relminmxid) compared to the corresponding
freeze-max-age setting, the number of update/deleted tuples
compared to the vacuum threshold, and the number of
inserted/updated/deleted tuples compared to the analyze threshold.
Once exception to the previous paragraph is for tables nearing
wraparound, i.e., those that have surpassed the effective failsafe
ages. In that case, the relfrozenxid/relminmxid-based score is
scaled aggressively so that the table has a decent chance of
sorting to the front of the list.
To adjust how strongly each component contributes to the score, the
following parameters can be adjusted from their default of 1.0 to
anywhere between 0.0 and 10.0 (inclusive). Setting all of these to
0.0 restores pre-v19 prioritization behavior:
autovacuum_freeze_score_weight
autovacuum_multixact_freeze_score_weight
autovacuum_vacuum_score_weight
autovacuum_vacuum_insert_score_weight
autovacuum_analyze_score_weight
This is intended to be a baby step towards smarter autovacuum
workers. Possible future improvements include, but are not limited
to, periodic reprioritization, automatic cost limit adjustments,
and better observability (e.g., a system view that shows current
scores). While we do not expect this commit to produce
earth-shattering improvements, it should in theory produce marginal
ones, and it is arguably a prerequisite for the aforementioned
follow-up changes.
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Reviewed-by: Greg Burd <greg@burd.me>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/aOaAuXREwnPZVISO%40nathan
---
doc/src/sgml/config.sgml | 90 ++++++
doc/src/sgml/maintenance.sgml | 103 +++++++
src/backend/postmaster/autovacuum.c | 283 +++++++++++++++---
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 481 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 75c22405a09..0d2a28207ed 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1062,6 +1062,109 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>. Note that
+ increasing this parameter's value also lowers the age at which this
+ component begins scaling aggressively, i.e., the scaling age is divided
+ by its value if greater than <literal>1.0</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>. Note
+ that increasing this parameter's value also lowers the age at which this
+ component begins scaling aggressively, i.e., the scaling age is divided
+ by its value if greater than <literal>1.0</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>. Otherwise,
+ these "weight" parameters are multiplied to their respective component
+ scores. For example, raising
+ <xref linkend="guc-autovacuum-analyze-score-weight"/> to
+ <literal>2.0</literal> effectively doubles the
+ <emphasis>analyze</emphasis> component score.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7ecb069c248..d695f1de4bd 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +316,30 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+/*
+ * This struct is used by relation_needs_vacanalyze() to return the table's
+ * score (i.e., the maximum of the component scores) as well as the component
+ * scores themselves.
+ */
+typedef struct
+{
+ double max; /* maximum of all values below */
+ double xid; /* transaction ID component */
+ double mxid; /* multixact ID component */
+ double vac; /* vacuum component */
+ double vac_ins; /* vacuum insert component */
+ double anl; /* analyze component */
+} AutoVacuumScores;
+
+/*
+ * This struct is used to track and sort the list of tables to process.
+ */
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +378,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ AutoVacuumScores *scores);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1896,19 @@ get_database_list(void)
return dblist;
}
+/*
+ * List comparator for TableToProcess. Note that this sorts the tables based
+ * on their scores in descending order.
+ */
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1922,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2034,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2075,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2142,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2175,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2310,19 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ /*
+ * In case list_sort() would modify the list even when all the scores are
+ * 0.0, skip sorting if all the weight parameters are set to 0.0. This is
+ * probably not necessary, but we want to ensure folks have a guaranteed
+ * escape hatch from the scoring system.
+ */
+ if (autovacuum_freeze_score_weight != 0.0 ||
+ autovacuum_multixact_freeze_score_weight != 0.0 ||
+ autovacuum_vacuum_score_weight != 0.0 ||
+ autovacuum_vacuum_insert_score_weight != 0.0 ||
+ autovacuum_analyze_score_weight != 0.0)
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2351,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2584,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2913,6 +2985,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ AutoVacuumScores scores;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2920,15 +2993,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &scores);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2969,6 +3039,47 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns scores that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum processes it. However, we have attempted to develop
+ * something that is expected to work for a large portion of workloads with
+ * reasonable parameter settings.
+ *
+ * The autovacuum table score is calculated as the maximum of the ratios of
+ * each of the table's relevant values to its threshold. For example, if the
+ * number of inserted tuples is 100, and the insert threshold for the table is
+ * 80, the insert score is 1.25. If all other scores are below that value, the
+ * returned score will be 1.25. The other criteria considered for the score
+ * are the table ages (both relfrozenxid and relminmxid) compared to the
+ * corresponding freeze-max-age setting, the number of updated/deleted tuples
+ * compared to the vacuum threshold, and the number of inserted/updated/deleted
+ * tuples compared to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the front of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 10.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
+ *
+ * The autovacuum table score is returned in scores->max. The component scores
+ * are also returned in the "scores" argument via the other members of the
+ * AutoVacuumScores struct.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2979,7 +3090,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ AutoVacuumScores *scores)
{
bool force_vacuum;
bool av_enabled;
@@ -3008,11 +3120,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ memset(scores, 0, sizeof(AutoVacuumScores));
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3060,17 +3177,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3079,13 +3196,65 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ scores->xid = (double) xid_age / freeze_max_age;
+ scores->mxid = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * begin scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ scores->xid = pow(scores->xid, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ scores->mxid = pow(scores->mxid, Max(1.0, (double) mxid_age / 100000000));
+
+ scores->xid *= autovacuum_freeze_score_weight;
+ scores->mxid *= autovacuum_multixact_freeze_score_weight;
+
+ scores->max = Max(scores->xid, scores->mxid);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3134,34 +3303,54 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ scores->vac = (double) vactuples / Max(vacthresh, 1);
+ scores->vac *= autovacuum_vacuum_score_weight;
+ scores->max = Max(scores->max, scores->vac);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ scores->vac_ins = (double) instuples / Max(vacinsthresh, 1);
+ scores->vac_ins *= autovacuum_vacuum_insert_score_weight;
+ scores->max = Max(scores->max, scores->vac_ins);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ scores->anl = (double) anltuples / Max(anlthresh, 1);
+ scores->anl *= autovacuum_analyze_score_weight;
+ scores->max = Max(scores->max, scores->anl);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (thresh %.0f, score %.2f), ins: %.0f (thresh %.0f, score %.2f), anl: %.0f (thresh %.0f, score %.2f), xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, scores->vac,
+ instuples, vacinsthresh, scores->vac_ins,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (thresh %.0f, score %.2f), ins: (disabled), anl: %.0f (thresh %.0f, score %.2f), xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ vactuples, vacthresh, scores->vac,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
}
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..cf15597385b 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0 # range 0.0-10.0
+#autovacuum_multixact_freeze_score_weight = 1.0 # range 0.0-10.0
+#autovacuum_vacuum_score_weight = 1.0 # range 0.0-10.0
+#autovacuum_vacuum_insert_score_weight = 1.0 # range 0.0-10.0
+#autovacuum_analyze_score_weight = 1.0 # range 0.0-10.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index decc9f7a572..a05b30dab5a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -187,6 +187,7 @@ AuthToken
AutoPrewarmReadStreamData
AutoPrewarmSharedState
AutoVacOpts
+AutoVacuumScores
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
@@ -3098,6 +3099,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v17-0001-Add-rudimentary-table-prioritization-to-autovacu.patch (35.7K, ../../acVjtJA2F68vdYTB@nathan/2-v17-0001-Add-rudimentary-table-prioritization-to-autovacu.patch)
download | inline diff:
From bf5d0ffc742ce744deb18ed91b8ce844b013eed5 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 25 Mar 2026 16:07:11 -0500
Subject: [PATCH v17 1/1] Add rudimentary table prioritization to autovacuum.
Autovacuum workers scan pg_class twice to collect the set of tables
to process. The first pass is for plain relations and materialized
views, and the second is for TOAST tables. When the worker finds a
table to process, it adds it to the end of a list. Later on, it
processes the tables in the same order as the list. This simple
strategy has worked surprisingly well for a long time, but there
have been many discussions over the years about trying to make it
better.
This commit introduces a scoring system that is used to sort the
aforementioned list of tables to process. The idea is to have
autovacuum workers prioritize tables that are furthest beyond their
thresholds (e.g., a table nearing transaction ID wraparound should
be vacuumed first). This prioritization scheme is certainly far
from perfect; there are simply too many possibilities for any
scoring technique to work across all workloads, and the situation
might change significantly between the time we calculate the score
and the time that autovacuum processes it. However, we have
attemped to develop something that is expected to work for a large
portion of workloads with reasonable parameter settings.
The score is calculated as the maximum of the ratios of each of the
table's relevant values to its threshold. For example, if the
number of inserted tuples is 100, and the insert threshold for the
table is 80, the insert score is 1.25. If all other scores are
below that value, the table's score will be 1.25. The other
criteria considered for the score are the table ages (both
relfrozenxid and relminmxid) compared to the corresponding
freeze-max-age setting, the number of update/deleted tuples
compared to the vacuum threshold, and the number of
inserted/updated/deleted tuples compared to the analyze threshold.
Once exception to the previous paragraph is for tables nearing
wraparound, i.e., those that have surpassed the effective failsafe
ages. In that case, the relfrozenxid/relminmxid-based score is
scaled aggressively so that the table has a decent chance of
sorting to the front of the list.
To adjust how strongly each component contributes to the score, the
following parameters can be adjusted from their default of 1.0 to
anywhere between 0.0 and 10.0 (inclusive). Setting all of these to
0.0 restores pre-v19 prioritization behavior:
autovacuum_freeze_score_weight
autovacuum_multixact_freeze_score_weight
autovacuum_vacuum_score_weight
autovacuum_vacuum_insert_score_weight
autovacuum_analyze_score_weight
This is intended to be a baby step towards smarter autovacuum
workers. Possible future improvements include, but are not limited
to, periodic reprioritization, automatic cost limit adjustments,
and better observability (e.g., a system view that shows current
scores). While we do not expect this commit to produce
earth-shattering improvements, it should in theory produce marginal
ones, and it is arguably a prerequisite for the aforementioned
follow-up changes.
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Reviewed-by: Greg Burd <greg@burd.me>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/aOaAuXREwnPZVISO%40nathan
---
doc/src/sgml/config.sgml | 90 ++++++
doc/src/sgml/maintenance.sgml | 103 +++++++
src/backend/postmaster/autovacuum.c | 283 +++++++++++++++---
src/backend/utils/misc/guc_parameters.dat | 40 +++
src/backend/utils/misc/postgresql.conf.sample | 5 +
src/include/postmaster/autovacuum.h | 6 +-
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 481 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8cdd826fbd3..229f41353eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9395,6 +9395,96 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-freeze-score-weight" xreflabel="autovacuum_freeze_score_weight">
+ <term><varname>autovacuum_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the transaction ID age component of
+ the score used by autovacuum for prioritization purposes. The default
+ is <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-multixact-freeze-score-weight" xreflabel="autovacuum_multixact_freeze_score_weight">
+ <term><varname>autovacuum_multixact_freeze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_multixact_freeze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the multixact ID age component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-score-weight" xreflabel="autovacuum_vacuum_score_weight">
+ <term><varname>autovacuum_vacuum_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-vacuum-insert-score-weight" xreflabel="autovacuum_vacuum_insert_score_weight">
+ <term><varname>autovacuum_vacuum_insert_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_vacuum_insert_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the vacuum insert threshold component
+ of the score used by autovacuum for prioritization purposes. The
+ default is <literal>1.0</literal>. This parameter can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-autovacuum-analyze-score-weight" xreflabel="autovacuum_analyze_score_weight">
+ <term><varname>autovacuum_analyze_score_weight</varname> (<type>floating point</type>)
+ <indexterm>
+ <primary><varname>autovacuum_analyze_score_weight</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the scaling factor of the analyze threshold component of the
+ score used by autovacuum for prioritization purposes. The default is
+ <literal>1.0</literal>. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command
+ line. See <xref linkend="autovacuum-priority"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 75c22405a09..0d2a28207ed 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -1062,6 +1062,109 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu
effectively prevent autovacuums from ever completing.
</para>
</warning>
+
+ <sect3 id="autovacuum-priority">
+ <title>Autovacuum Prioritization</title>
+
+ <para>
+ Autovacuum decides what to process in two steps: first it chooses a
+ database, then it chooses the tables within that database. The autovacuum
+ launcher process prioritizes databases at risk of transaction ID or
+ multixact ID wraparound, else it chooses the database processed least
+ recently. As an exception, it skips databases with no connections or no
+ activity since the last statistics reset, unless at risk of wraparound.
+ </para>
+
+ <para>
+ Within a database, the autovacuum worker process builds a list of tables
+ that require vacuum or analyze and sorts them using a scoring system. It
+ scores each table by taking the maximum value of several component scores
+ representing various criteria important to vacuum or analyze. Those
+ components are as follows:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>transaction ID</emphasis> component measures the age in
+ transactions of the table's
+ <structname>pg_class</structname>.<structfield>relfrozenxid</structfield>
+ field as compared to <xref linkend="guc-autovacuum-freeze-max-age"/>.
+ Furthermore, this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-failsafe-age"/>. The final value for this
+ component can be adjusted via
+ <xref linkend="guc-autovacuum-freeze-score-weight"/>. Note that
+ increasing this parameter's value also lowers the age at which this
+ component begins scaling aggressively, i.e., the scaling age is divided
+ by its value if greater than <literal>1.0</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>multixact ID</emphasis> component measures the age in
+ multixacts of the table's
+ <structname>pg_class</structname>.<structfield>relminmxid</structfield>
+ field as compared to
+ <xref linkend="guc-autovacuum-multixact-freeze-max-age"/>. Furthermore,
+ this component increases greatly once the age surpasses
+ <xref linkend="guc-vacuum-multixact-failsafe-age"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-multixact-freeze-score-weight"/>. Note
+ that increasing this parameter's value also lowers the age at which this
+ component begins scaling aggressively, i.e., the scaling age is divided
+ by its value if greater than <literal>1.0</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum</emphasis> component measures the number of updated
+ or deleted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-threshold"/>,
+ <xref linkend="guc-autovacuum-vacuum-scale-factor"/>, and
+ <xref linkend="guc-autovacuum-vacuum-max-threshold"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>vacuum insert</emphasis> component measures the number of
+ inserted tuples as compared to the threshold calculated with
+ <xref linkend="guc-autovacuum-vacuum-insert-threshold"/> and
+ <xref linkend="guc-autovacuum-vacuum-insert-scale-factor"/>. The final
+ value for this component can be adjusted via
+ <xref linkend="guc-autovacuum-vacuum-insert-score-weight"/>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <emphasis>analyze</emphasis> component measures the number of
+ inserted, updated, or deleted tuples as compared to the threshold
+ calculated with
+ <xref linkend="guc-autovacuum-analyze-threshold"/> and
+ <xref linkend="guc-autovacuum-analyze-scale-factor"/>. The final value
+ for this component can be adjusted via
+ <xref linkend="guc-autovacuum-analyze-score-weight"/>.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ To revert to the prioritization strategy used before
+ <productname>PostgreSQL</productname> 19 (i.e., the order the tables are
+ listed in the <literal>pg_class</literal> system catalog), set all of the
+ aforementioned "weight" parameters to <literal>0.0</literal>. Otherwise,
+ these "weight" parameters are multiplied to their respective component
+ scores. For example, raising
+ <xref linkend="guc-autovacuum-analyze-score-weight"/> to
+ <literal>2.0</literal> effectively doubles the
+ <emphasis>analyze</emphasis> component score.
+ </para>
+ </sect3>
</sect2>
</sect1>
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7ecb069c248..d695f1de4bd 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -62,6 +62,7 @@
*/
#include "postgres.h"
+#include <math.h>
#include <signal.h>
#include <sys/time.h>
#include <unistd.h>
@@ -130,7 +131,11 @@ int autovacuum_anl_thresh;
double autovacuum_anl_scale;
int autovacuum_freeze_max_age;
int autovacuum_multixact_freeze_max_age;
-
+double autovacuum_freeze_score_weight = 1.0;
+double autovacuum_multixact_freeze_score_weight = 1.0;
+double autovacuum_vacuum_score_weight = 1.0;
+double autovacuum_vacuum_insert_score_weight = 1.0;
+double autovacuum_analyze_score_weight = 1.0;
double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
@@ -311,6 +316,30 @@ static AutoVacuumShmemStruct *AutoVacuumShmem;
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+/*
+ * This struct is used by relation_needs_vacanalyze() to return the table's
+ * score (i.e., the maximum of the component scores) as well as the component
+ * scores themselves.
+ */
+typedef struct
+{
+ double max; /* maximum of all values below */
+ double xid; /* transaction ID component */
+ double mxid; /* multixact ID component */
+ double vac; /* vacuum component */
+ double vac_ins; /* vacuum insert component */
+ double anl; /* analyze component */
+} AutoVacuumScores;
+
+/*
+ * This struct is used to track and sort the list of tables to process.
+ */
+typedef struct
+{
+ Oid oid;
+ double score;
+} TableToProcess;
+
/*
* Dummy pointer to persuade Valgrind that we've not leaked the array of
* avl_dbase structs. Make it global to ensure the compiler doesn't
@@ -349,7 +378,8 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
- bool *dovacuum, bool *doanalyze, bool *wraparound);
+ bool *dovacuum, bool *doanalyze, bool *wraparound,
+ AutoVacuumScores *scores);
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
@@ -1866,6 +1896,19 @@ get_database_list(void)
return dblist;
}
+/*
+ * List comparator for TableToProcess. Note that this sorts the tables based
+ * on their scores in descending order.
+ */
+static int
+TableToProcessComparator(const ListCell *a, const ListCell *b)
+{
+ TableToProcess *t1 = (TableToProcess *) lfirst(a);
+ TableToProcess *t2 = (TableToProcess *) lfirst(b);
+
+ return (t2->score < t1->score) ? -1 : (t2->score > t1->score) ? 1 : 0;
+}
+
/*
* Process a database table-by-table
*
@@ -1879,7 +1922,7 @@ do_autovacuum(void)
HeapTuple tuple;
TableScanDesc relScan;
Form_pg_database dbForm;
- List *table_oids = NIL;
+ List *tables_to_process = NIL;
List *orphan_oids = NIL;
HASHCTL ctl;
HTAB *table_toast_map;
@@ -1991,6 +2034,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
if (classForm->relkind != RELKIND_RELATION &&
classForm->relkind != RELKIND_MATVIEW)
@@ -2031,11 +2075,18 @@ do_autovacuum(void)
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
- /* Relations that need work are added to table_oids */
+ /* Relations that need work are added to tables_to_process */
if (dovacuum || doanalyze)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/*
* Remember TOAST associations for the second pass. Note: we must do
@@ -2091,6 +2142,7 @@ do_autovacuum(void)
bool dovacuum;
bool doanalyze;
bool wraparound;
+ AutoVacuumScores scores;
/*
* We cannot safely process other backends' temp tables, so skip 'em.
@@ -2123,11 +2175,18 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- &dovacuum, &doanalyze, &wraparound);
+ &dovacuum, &doanalyze, &wraparound,
+ &scores);
/* ignore analyze for toast tables */
if (dovacuum)
- table_oids = lappend_oid(table_oids, relid);
+ {
+ TableToProcess *table = palloc_object(TableToProcess);
+
+ table->oid = relid;
+ table->score = scores.max;
+ tables_to_process = lappend(tables_to_process, table);
+ }
/* Release stuff to avoid leakage */
if (free_relopts)
@@ -2251,6 +2310,19 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
}
+ /*
+ * In case list_sort() would modify the list even when all the scores are
+ * 0.0, skip sorting if all the weight parameters are set to 0.0. This is
+ * probably not necessary, but we want to ensure folks have a guaranteed
+ * escape hatch from the scoring system.
+ */
+ if (autovacuum_freeze_score_weight != 0.0 ||
+ autovacuum_multixact_freeze_score_weight != 0.0 ||
+ autovacuum_vacuum_score_weight != 0.0 ||
+ autovacuum_vacuum_insert_score_weight != 0.0 ||
+ autovacuum_analyze_score_weight != 0.0)
+ list_sort(tables_to_process, TableToProcessComparator);
+
/*
* Optionally, create a buffer access strategy object for VACUUM to use.
* We use the same BufferAccessStrategy object for all tables VACUUMed by
@@ -2279,9 +2351,9 @@ do_autovacuum(void)
/*
* Perform operations on collected tables.
*/
- foreach(cell, table_oids)
+ foreach_ptr(TableToProcess, table, tables_to_process)
{
- Oid relid = lfirst_oid(cell);
+ Oid relid = table->oid;
HeapTuple classTup;
autovac_table *tab;
bool isshared;
@@ -2512,7 +2584,7 @@ deleted:
pg_atomic_test_set_flag(&MyWorkerInfo->wi_dobalance);
}
- list_free(table_oids);
+ list_free_deep(tables_to_process);
/*
* Perform additional work items, as requested by backends.
@@ -2913,6 +2985,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
+ AutoVacuumScores scores;
/* fetch the pgstat table entry */
tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared,
@@ -2920,15 +2993,12 @@ recheck_relation_needs_vacanalyze(Oid relid,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
- dovacuum, doanalyze, wraparound);
+ dovacuum, doanalyze, wraparound,
+ &scores);
/* Release tabentry to avoid leakage */
if (tabentry)
pfree(tabentry);
-
- /* ignore ANALYZE for toast tables */
- if (classForm->relkind == RELKIND_TOASTVALUE)
- *doanalyze = false;
}
/*
@@ -2969,6 +3039,47 @@ recheck_relation_needs_vacanalyze(Oid relid,
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
+ *
+ * This function also returns scores that can be used to sort the list of
+ * tables to process. The idea is to have autovacuum prioritize tables that
+ * are furthest beyond their thresholds (e.g., a table nearing transaction ID
+ * wraparound should be vacuumed first). This prioritization scheme is
+ * certainly far from perfect; there are simply too many possibilities for any
+ * scoring technique to work across all workloads, and the situation might
+ * change significantly between the time we calculate the score and the time
+ * that autovacuum processes it. However, we have attempted to develop
+ * something that is expected to work for a large portion of workloads with
+ * reasonable parameter settings.
+ *
+ * The autovacuum table score is calculated as the maximum of the ratios of
+ * each of the table's relevant values to its threshold. For example, if the
+ * number of inserted tuples is 100, and the insert threshold for the table is
+ * 80, the insert score is 1.25. If all other scores are below that value, the
+ * returned score will be 1.25. The other criteria considered for the score
+ * are the table ages (both relfrozenxid and relminmxid) compared to the
+ * corresponding freeze-max-age setting, the number of updated/deleted tuples
+ * compared to the vacuum threshold, and the number of inserted/updated/deleted
+ * tuples compared to the analyze threshold.
+ *
+ * One exception to the previous paragraph is for tables nearing wraparound,
+ * i.e., those that have surpassed the effective failsafe ages. In that case,
+ * the relfrozen/relminmxid-based score is scaled aggressively so that the
+ * table has a decent chance of sorting to the front of the list.
+ *
+ * To adjust how strongly each component contributes to the score, the
+ * following parameters can be adjusted from their default of 1.0 to anywhere
+ * between 0.0 and 10.0 (inclusive). Setting all of these to 0.0 restores
+ * pre-v19 prioritization behavior:
+ *
+ * autovacuum_freeze_score_weight
+ * autovacuum_multixact_freeze_score_weight
+ * autovacuum_vacuum_score_weight
+ * autovacuum_vacuum_insert_score_weight
+ * autovacuum_analyze_score_weight
+ *
+ * The autovacuum table score is returned in scores->max. The component scores
+ * are also returned in the "scores" argument via the other members of the
+ * AutoVacuumScores struct.
*/
static void
relation_needs_vacanalyze(Oid relid,
@@ -2979,7 +3090,8 @@ relation_needs_vacanalyze(Oid relid,
/* output params below */
bool *dovacuum,
bool *doanalyze,
- bool *wraparound)
+ bool *wraparound,
+ AutoVacuumScores *scores)
{
bool force_vacuum;
bool av_enabled;
@@ -3008,11 +3120,16 @@ relation_needs_vacanalyze(Oid relid,
int multixact_freeze_max_age;
TransactionId xidForceLimit;
TransactionId relfrozenxid;
+ MultiXactId relminmxid;
MultiXactId multiForceLimit;
Assert(classForm != NULL);
Assert(OidIsValid(relid));
+ memset(scores, 0, sizeof(AutoVacuumScores));
+ *dovacuum = false;
+ *doanalyze = false;
+
/*
* Determine vacuum/analyze equation parameters. We have two possible
* sources: the passed reloptions (which could be a main table or a toast
@@ -3060,17 +3177,17 @@ relation_needs_vacanalyze(Oid relid,
av_enabled = (relopts ? relopts->enabled : true);
+ relfrozenxid = classForm->relfrozenxid;
+ relminmxid = classForm->relminmxid;
+
/* Force vacuum if table is at risk of wraparound */
xidForceLimit = recentXid - freeze_max_age;
if (xidForceLimit < FirstNormalTransactionId)
xidForceLimit -= FirstNormalTransactionId;
- relfrozenxid = classForm->relfrozenxid;
force_vacuum = (TransactionIdIsNormal(relfrozenxid) &&
TransactionIdPrecedes(relfrozenxid, xidForceLimit));
if (!force_vacuum)
{
- MultiXactId relminmxid = classForm->relminmxid;
-
multiForceLimit = recentMulti - multixact_freeze_max_age;
if (multiForceLimit < FirstMultiXactId)
multiForceLimit -= FirstMultiXactId;
@@ -3079,13 +3196,65 @@ relation_needs_vacanalyze(Oid relid,
}
*wraparound = force_vacuum;
+ /* Update the score. */
+ if (force_vacuum)
+ {
+ uint32 xid_age;
+ uint32 mxid_age;
+ int effective_xid_failsafe_age;
+ int effective_mxid_failsafe_age;
+
+ /*
+ * To calculate the (M)XID age portion of the score, divide the age by
+ * its respective *_freeze_max_age parameter.
+ */
+ xid_age = TransactionIdIsNormal(relfrozenxid) ? recentXid - relfrozenxid : 0;
+ mxid_age = MultiXactIdIsValid(relminmxid) ? recentMulti - relminmxid : 0;
+
+ scores->xid = (double) xid_age / freeze_max_age;
+ scores->mxid = (double) mxid_age / multixact_freeze_max_age;
+
+ /*
+ * To ensure tables are given increased priority once they begin
+ * approaching wraparound, we scale the score aggressively if the ages
+ * surpass vacuum_failsafe_age or vacuum_multixact_failsafe_age.
+ *
+ * As in vacuum_xid_failsafe_check(), the effective failsafe age is no
+ * less than 105% the value of the respective *_freeze_max_age
+ * parameter. Note that per-table settings could result in a low
+ * score even if the table surpasses the failsafe settings. However,
+ * this is a strange enough corner case that we don't bother trying to
+ * handle it.
+ *
+ * We further adjust the effective failsafe ages with the weight
+ * parameters so that increasing them lowers the ages at which we
+ * begin scaling aggressively.
+ */
+ effective_xid_failsafe_age = Max(vacuum_failsafe_age,
+ autovacuum_freeze_max_age * 1.05);
+ effective_mxid_failsafe_age = Max(vacuum_multixact_failsafe_age,
+ autovacuum_multixact_freeze_max_age * 1.05);
+
+ if (autovacuum_freeze_score_weight > 1.0)
+ effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
+ if (autovacuum_multixact_freeze_score_weight > 1.0)
+ effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
+
+ if (xid_age >= effective_xid_failsafe_age)
+ scores->xid = pow(scores->xid, Max(1.0, (double) xid_age / 100000000));
+ if (mxid_age >= effective_mxid_failsafe_age)
+ scores->mxid = pow(scores->mxid, Max(1.0, (double) mxid_age / 100000000));
+
+ scores->xid *= autovacuum_freeze_score_weight;
+ scores->mxid *= autovacuum_multixact_freeze_score_weight;
+
+ scores->max = Max(scores->xid, scores->mxid);
+ *dovacuum = true;
+ }
+
/* User disabled it in pg_class.reloptions? (But ignore if at risk) */
if (!av_enabled && !force_vacuum)
- {
- *doanalyze = false;
- *dovacuum = false;
return;
- }
/*
* If we found stats for the table, and autovacuum is currently enabled,
@@ -3134,34 +3303,54 @@ relation_needs_vacanalyze(Oid relid,
vac_ins_scale_factor * reltuples * pcnt_unfrozen;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ /*
+ * Determine if this table needs vacuum, and update the score if it
+ * does.
+ */
+ if (vactuples > vacthresh)
+ {
+ scores->vac = (double) vactuples / Max(vacthresh, 1);
+ scores->vac *= autovacuum_vacuum_score_weight;
+ scores->max = Max(scores->max, scores->vac);
+ *dovacuum = true;
+ }
+
+ if (vac_ins_base_thresh >= 0 && instuples > vacinsthresh)
+ {
+ scores->vac_ins = (double) instuples / Max(vacinsthresh, 1);
+ scores->vac_ins *= autovacuum_vacuum_insert_score_weight;
+ scores->max = Max(scores->max, scores->vac_ins);
+ *dovacuum = true;
+ }
+
+ /*
+ * Determine if this table needs analyze, and update the score if it
+ * does. Note that we don't analyze TOAST tables and pg_statistic.
+ */
+ if (anltuples > anlthresh &&
+ relid != StatisticRelationId &&
+ classForm->relkind != RELKIND_TOASTVALUE)
+ {
+ scores->anl = (double) anltuples / Max(anlthresh, 1);
+ scores->anl *= autovacuum_analyze_score_weight;
+ scores->max = Max(scores->max, scores->anl);
+ *doanalyze = true;
+ }
+
if (vac_ins_base_thresh >= 0)
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (thresh %.0f, score %.2f), ins: %.0f (thresh %.0f, score %.2f), anl: %.0f (thresh %.0f, score %.2f), xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, instuples, vacinsthresh, anltuples, anlthresh);
+ vactuples, vacthresh, scores->vac,
+ instuples, vacinsthresh, scores->vac_ins,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
else
- elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), ins: (disabled), anl: %.0f (threshold %.0f)",
+ elog(DEBUG3, "%s: vac: %.0f (thresh %.0f, score %.2f), ins: (disabled), anl: %.0f (thresh %.0f, score %.2f), xid score: %.2f, mxid score: %.2f",
NameStr(classForm->relname),
- vactuples, vacthresh, anltuples, anlthresh);
-
- /* Determine if this table needs vacuum or analyze. */
- *dovacuum = force_vacuum || (vactuples > vacthresh) ||
- (vac_ins_base_thresh >= 0 && instuples > vacinsthresh);
- *doanalyze = (anltuples > anlthresh);
+ vactuples, vacthresh, scores->vac,
+ anltuples, anlthresh, scores->anl,
+ scores->xid, scores->mxid);
}
- else
- {
- /*
- * Skip a table not found in stat hash, unless we have to force vacuum
- * for anti-wrap purposes. If it's not acted upon, there's no need to
- * vacuum it.
- */
- *dovacuum = force_vacuum;
- *doanalyze = false;
- }
-
- /* ANALYZE refuses to work with pg_statistic */
- if (relid == StatisticRelationId)
- *doanalyze = false;
}
/*
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0c9854ad8fc..0a862693fcd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -136,6 +136,14 @@
max => '100.0',
},
+{ name => 'autovacuum_analyze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of analyze score for autovacuum prioritization.',
+ variable => 'autovacuum_analyze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_analyze_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts, updates, or deletes prior to analyze.',
variable => 'autovacuum_anl_thresh',
@@ -154,6 +162,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
@@ -171,6 +187,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_multixact_freeze_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of multixact freeze score for autovacuum prioritization.',
+ variable => 'autovacuum_multixact_freeze_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_naptime', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Time to sleep between autovacuum runs.',
flags => 'GUC_UNIT_S',
@@ -207,6 +231,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_insert_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum insert score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_insert_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_insert_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple inserts prior to vacuum.',
long_desc => '-1 disables insert vacuums.',
@@ -233,6 +265,14 @@
max => '100.0',
},
+{ name => 'autovacuum_vacuum_score_weight', type => 'real', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Scaling factor of vacuum score for autovacuum prioritization.',
+ variable => 'autovacuum_vacuum_score_weight',
+ boot_val => '1.0',
+ min => '0.0',
+ max => '10.0',
+},
+
{ name => 'autovacuum_vacuum_threshold', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Minimum number of tuple updates or deletes prior to vacuum.',
variable => 'autovacuum_vac_thresh',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e4abe6c0077..cf15597385b 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -733,6 +733,11 @@
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
+#autovacuum_freeze_score_weight = 1.0 # range 0.0-10.0
+#autovacuum_multixact_freeze_score_weight = 1.0 # range 0.0-10.0
+#autovacuum_vacuum_score_weight = 1.0 # range 0.0-10.0
+#autovacuum_vacuum_insert_score_weight = 1.0 # range 0.0-10.0
+#autovacuum_analyze_score_weight = 1.0 # range 0.0-10.0
#autovacuum_vacuum_cost_delay = 2ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..b21d111d4d5 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -43,7 +43,11 @@ extern PGDLLIMPORT int autovacuum_freeze_max_age;
extern PGDLLIMPORT int autovacuum_multixact_freeze_max_age;
extern PGDLLIMPORT double autovacuum_vac_cost_delay;
extern PGDLLIMPORT int autovacuum_vac_cost_limit;
-
+extern PGDLLIMPORT double autovacuum_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_multixact_freeze_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_score_weight;
+extern PGDLLIMPORT double autovacuum_vacuum_insert_score_weight;
+extern PGDLLIMPORT double autovacuum_analyze_score_weight;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
extern PGDLLIMPORT int Log_autoanalyze_min_duration;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index decc9f7a572..a05b30dab5a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -187,6 +187,7 @@ AuthToken
AutoPrewarmReadStreamData
AutoPrewarmSharedState
AutoVacOpts
+AutoVacuumScores
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
@@ -3098,6 +3099,7 @@ TableScanDesc
TableScanDescData
TableSpaceCacheEntry
TableSpaceOpts
+TableToProcess
TablespaceList
TablespaceListCell
TapeBlockTrailer
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-26 21:29 ` David Rowley <dgrowleyml@gmail.com>
2026-03-27 15:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2026-03-26 21:29 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, 27 Mar 2026 at 05:49, Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> On Thu, Mar 26, 2026 at 01:28:16PM +1300, David Rowley wrote:
> > A review:
>
> Thanks. I believe I've addressed all your feedback.
It might just be a personal taste thing, but I'd have done the
following differently:
+ if (autovacuum_freeze_score_weight != 0.0 ||
+ autovacuum_multixact_freeze_score_weight != 0.0 ||
+ autovacuum_vacuum_score_weight != 0.0 ||
+ autovacuum_vacuum_insert_score_weight != 0.0 ||
+ autovacuum_analyze_score_weight != 0.0)
+ list_sort(tables_to_process, TableToProcessComparator);
I'd have done:
+ table->score = scores.max;
+ sort_required |= (scores.max != 0.0);
+ tables_to_process = lappend(tables_to_process, table);
...
if (sort_required)
list_sort(tables_to_process, TableToProcessComparator);
But, I'm fine if you'd rather keep it the way you have it.
No further comments at this stage.
Thanks for working on this.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2026-03-27 15:18 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-27 23:17 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-03-27 15:18 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Sami Imseih <samimseih@gmail.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Fri, Mar 27, 2026 at 10:29:47AM +1300, David Rowley wrote:
> Thanks for working on this.
Committed!
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-27 15:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-03-27 23:17 ` Sami Imseih <samimseih@gmail.com>
2026-08-31 15:23 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2026-03-27 23:17 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> Committed!
Thanks for pushing this!
Also, [1] for the priority view discussed earlier [2].
[1] https://www.postgresql.org/message-id/CAA5RZ0s4xjMrB-VAnLccC7kY8d0-4806-Lsac-czJsdA1LXtAw%40mail.gma...
[2] [https://www.postgresql.org/message-id/CAApHDvqQN-B2sQov8nsfZOmx-VeJMauSf4kLa3A8LsK1tUyBNw%40mail.gma...]
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-27 15:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-27 23:17 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2026-08-31 15:23 ` Nathan Bossart <nathandbossart@gmail.com>
2026-08-31 15:49 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-08-31 15:23 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Claude noticed a corner case that needs handling. In short, if the freeze
score weights are high enough, the pow() calls can have the opposite of the
intended effect, since the freeze scores may be less than 1. Patch
attached.
--
nathan
From 2b2942e38db24d84aeb17a9075d4173b2b797fa9 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Mon, 31 Aug 2026 10:09:37 -0500
Subject: [PATCH v1 1/1] Fix backwards scaling of autovacuum freeze scores.
Presently, once a table's age passes the effective failsafe age,
relation_needs_vacanalyze() raises the ratio of that age to the
freeze-max-age to a power of at least 1.0, so that the table sorts
towards the front of the list. That amplifies the score only
while the ratio is above 1.0. Below it, exponentiation shrinks
the score, and shrinks it further as the age grows, so a table can
cross into the failsafe range and see its score fall, and keep
falling as it ages. Ratios below 1.0 are reachable because the
effective failsafe age is divided by
autovacuum_freeze_score_weight, which can leave it under the
freeze-max-age. With autovacuum_freeze_max_age at 2000000000 and
the weight at 10.0, tables aged 220M, 400M, and 736M score 0.078,
0.016, and 0.0064, which is exactly backwards, and all three sit
below what they would score with no scaling at all.
To fix, only scale a score that the exponentiation will actually
amplify. Raising the weight still starts the scaling earlier,
just not so early that it does the opposite of what it is for.
The default configuration is unaffected, since there scaling
begins at an age of 1.6 billion, where the ratio is already 8.0.
Oversight in commit d7965d65fc.
Backpatch-through: 19
---
src/backend/postmaster/autovacuum.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 393e81d53e9..13d330913be 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -3210,9 +3210,15 @@ relation_needs_vacanalyze(Oid relid,
if (autovacuum_multixact_freeze_score_weight > 1.0)
effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
- if (xid_age >= effective_xid_failsafe_age)
+ /*
+ * Note that raising a score below 1.0 to a power greater than 1.0 shrinks
+ * it, and shrinks it further as the age grows, so scaling such a score
+ * would leave an older table sorting behind a younger one. Only scale
+ * scores that the exponentiation will actually amplify.
+ */
+ if (xid_age >= effective_xid_failsafe_age && scores->xid > 1.0)
scores->xid = pow(scores->xid, Max(1.0, (double) xid_age / 100000000));
- if (mxid_age >= effective_mxid_failsafe_age)
+ if (mxid_age >= effective_mxid_failsafe_age && scores->mxid > 1.0)
scores->mxid = pow(scores->mxid, Max(1.0, (double) mxid_age / 100000000));
scores->xid *= autovacuum_freeze_score_weight;
--
2.55.0
Attachments:
[text/plain] v1-0001-Fix-backwards-scaling-of-autovacuum-freeze-scores.patch (2.6K, ../../apWcbq91BAfhmsKU@nathan/2-v1-0001-Fix-backwards-scaling-of-autovacuum-freeze-scores.patch)
download | inline diff:
From 2b2942e38db24d84aeb17a9075d4173b2b797fa9 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Mon, 31 Aug 2026 10:09:37 -0500
Subject: [PATCH v1 1/1] Fix backwards scaling of autovacuum freeze scores.
Presently, once a table's age passes the effective failsafe age,
relation_needs_vacanalyze() raises the ratio of that age to the
freeze-max-age to a power of at least 1.0, so that the table sorts
towards the front of the list. That amplifies the score only
while the ratio is above 1.0. Below it, exponentiation shrinks
the score, and shrinks it further as the age grows, so a table can
cross into the failsafe range and see its score fall, and keep
falling as it ages. Ratios below 1.0 are reachable because the
effective failsafe age is divided by
autovacuum_freeze_score_weight, which can leave it under the
freeze-max-age. With autovacuum_freeze_max_age at 2000000000 and
the weight at 10.0, tables aged 220M, 400M, and 736M score 0.078,
0.016, and 0.0064, which is exactly backwards, and all three sit
below what they would score with no scaling at all.
To fix, only scale a score that the exponentiation will actually
amplify. Raising the weight still starts the scaling earlier,
just not so early that it does the opposite of what it is for.
The default configuration is unaffected, since there scaling
begins at an age of 1.6 billion, where the ratio is already 8.0.
Oversight in commit d7965d65fc.
Backpatch-through: 19
---
src/backend/postmaster/autovacuum.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 393e81d53e9..13d330913be 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -3210,9 +3210,15 @@ relation_needs_vacanalyze(Oid relid,
if (autovacuum_multixact_freeze_score_weight > 1.0)
effective_mxid_failsafe_age /= autovacuum_multixact_freeze_score_weight;
- if (xid_age >= effective_xid_failsafe_age)
+ /*
+ * Note that raising a score below 1.0 to a power greater than 1.0 shrinks
+ * it, and shrinks it further as the age grows, so scaling such a score
+ * would leave an older table sorting behind a younger one. Only scale
+ * scores that the exponentiation will actually amplify.
+ */
+ if (xid_age >= effective_xid_failsafe_age && scores->xid > 1.0)
scores->xid = pow(scores->xid, Max(1.0, (double) xid_age / 100000000));
- if (mxid_age >= effective_mxid_failsafe_age)
+ if (mxid_age >= effective_mxid_failsafe_age && scores->mxid > 1.0)
scores->mxid = pow(scores->mxid, Max(1.0, (double) mxid_age / 100000000));
scores->xid *= autovacuum_freeze_score_weight;
--
2.55.0
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-27 15:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-27 23:17 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-08-31 15:23 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-08-31 15:49 ` Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-08-31 16:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Bharath Rupireddy @ 2026-08-31 15:49 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Hi,
On Mon, Aug 31, 2026 at 8:23 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> Claude noticed a corner case that needs handling. In short, if the freeze
> score weights are high enough, the pow() calls can have the opposite of the
> intended effect, since the freeze scores may be less than 1. Patch
> attached.
Nice catch! Yes, it seems like an obvious issue and the fix looks good to me.
Can we add some tests for HEAD for the autovacuum scoring feature? I
understand it's a bit hard to get the tests to run deterministically,
but if it's worth it, I can give it a try.
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-27 15:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-27 23:17 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-08-31 15:23 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-08-31 15:49 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
@ 2026-08-31 16:47 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-01 05:18 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-08-31 16:47 UTC (permalink / raw)
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Greg Burd <greg@burd.me>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Mon, Aug 31, 2026 at 08:49:08AM -0700, Bharath Rupireddy wrote:
> On Mon, Aug 31, 2026 at 8:23 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
>> Claude noticed a corner case that needs handling. In short, if the freeze
>> score weights are high enough, the pow() calls can have the opposite of the
>> intended effect, since the freeze scores may be less than 1. Patch
>> attached.
>
> Nice catch! Yes, it seems like an obvious issue and the fix looks good to me.
Thanks for looking.
> Can we add some tests for HEAD for the autovacuum scoring feature? I
> understand it's a bit hard to get the tests to run deterministically,
> but if it's worth it, I can give it a try.
Seems worth a try.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-27 15:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-27 23:17 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-08-31 15:23 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-08-31 15:49 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-08-31 16:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-09-01 05:18 ` Greg Burd <greg@burd.me>
2026-09-01 20:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Greg Burd @ 2026-09-01 05:18 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Mon, Aug 31, 2026 at 10:23:26AM -0500, Nathan Bossart wrote:
> Claude noticed a corner case that needs handling. In short, if the freeze
> score weights are high enough, the pow() calls can have the opposite of the
> intended effect, since the freeze scores may be less than 1. Patch
> attached.
Hey Nathan, Bharath,
Confirmed, and +1 on the fix. I worked through the arithmetic and it's a
real oversight from d7965d65fc which I reviewed, apologies for not spotting
this back then.
It seems the trigger is that the weight divides effective_xid_failsafe_age:
if (autovacuum_freeze_score_weight > 1.0)
effective_xid_failsafe_age /= autovacuum_freeze_score_weight;
so a large enough weight can pull the failsafe threshold below
freeze_max_age, i.e. down into the region where
scores->xid = xid_age / freeze_max_age
is still < 1.0. Once we're there, pow(base < 1.0, exponent > 1.0) shrinks
the score, and shrinks it further as the exponent (age/1e8) grows. So a
table's score falls the moment it crosses into the failsafe range and keeps
falling as it ages, exactly backwards from the intent.
Reproducing your example numbers (freeze_max_age = 2000000000, weight = 10.0,
so effective_xid_failsafe_age = max(1.6e9, 2.1e9)/10 = 210M):
age before after
220M 0.077815 1.10
400M 0.016000 2.00
736M 0.006377 3.68
Before the patch all three sat below their unscaled scores and, worse,
decreased with age. After the patch the sub-1.0 bases skip pow() entirely
and fall back to ratio * weight (0.11*10 = 1.1, etc.), so they increase
monotonically again, as intended.
best,
-greg
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-27 15:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-27 23:17 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-08-31 15:23 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-08-31 15:49 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-08-31 16:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-09-01 05:18 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
@ 2026-09-01 20:22 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-02 01:19 ` Re: another autovacuum scheduling thread shihao zhong <zhong950419@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2026-09-01 20:22 UTC (permalink / raw)
To: Greg Burd <greg@burd.me>; +Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Tue, Sep 01, 2026 at 01:18:42AM -0400, Greg Burd wrote:
> Confirmed, and +1 on the fix. I worked through the arithmetic and it's a
> real oversight from d7965d65fc which I reviewed, apologies for not spotting
> this back then.
Committed, thanks for looking.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 15:53 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 11:44 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-20 04:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-22 00:08 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Re: another autovacuum scheduling thread Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:12 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 22:00 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2026-03-27 15:18 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-03-27 23:17 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2026-08-31 15:23 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-08-31 15:49 ` Re: another autovacuum scheduling thread Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-08-31 16:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2026-09-01 05:18 ` Re: another autovacuum scheduling thread Greg Burd <greg@burd.me>
2026-09-01 20:22 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2026-09-02 01:19 ` shihao zhong <zhong950419@gmail.com>
0 siblings, 0 replies; 150+ messages in thread
From: shihao zhong @ 2026-09-02 01:19 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Greg Burd <greg@burd.me>; Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; Sami Imseih <samimseih@gmail.com>; David Rowley <dgrowleyml@gmail.com>; Jim Nasby <jnasby@upgrade.com>; Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
Hi,
Thanks for committing the fix, Nathan.
> Can we add some tests for HEAD for the autovacuum scoring feature? I
> understand it's a bit hard to get the tests to run deterministically,
> but if it's worth it, I can give it a try.
I already had one written, so let me share it to save you the effort.
To keep it deterministic and fast, it fakes an old relfrozenxid and
relminmxid with a catalog UPDATE instead of consuming transactions.
Patch attached, on top of your commit.
Thanks,
Shihao
Attachments:
[application/octet-stream] 0001-Add-a-test-for-autovacuum-freeze-score-scaling.patch (4.7K, ../../CAGRkXqSnGfeZNB-k4bwg=j88r83yz=AJ0aKAGdYbzczMeDwexA@mail.gmail.com/2-0001-Add-a-test-for-autovacuum-freeze-score-scaling.patch)
download | inline diff:
From e991848607bd2dee6cc92101458cebd26ec1d148 Mon Sep 17 00:00:00 2001
From: Zhong ShiHao <zhong950419@gmail.com>
Date: Tue, 1 Sep 2026 21:16:43 -0400
Subject: [PATCH] Add a test for autovacuum freeze score scaling
Assert that raising an autovacuum freeze weight never lowers the freeze
score, for both the xid and multixact components. The test fakes an old
relfrozenxid/relminmxid (safe: the score reads them only arithmetically,
never via clog) so it runs in under a second with no PG_TEST_EXTRA gate.
---
src/test/modules/test_autovacuum/meson.build | 1 +
.../test_autovacuum/t/003_freeze_score.pl | 85 +++++++++++++++++++
2 files changed, 86 insertions(+)
create mode 100644 src/test/modules/test_autovacuum/t/003_freeze_score.pl
diff --git a/src/test/modules/test_autovacuum/meson.build b/src/test/modules/test_autovacuum/meson.build
index 970b9aaae4b..56601789960 100644
--- a/src/test/modules/test_autovacuum/meson.build
+++ b/src/test/modules/test_autovacuum/meson.build
@@ -11,6 +11,7 @@ tests += {
'tests': [
't/001_parallel_autovacuum.pl',
't/002_toast_relopts.pl',
+ 't/003_freeze_score.pl',
],
},
}
diff --git a/src/test/modules/test_autovacuum/t/003_freeze_score.pl b/src/test/modules/test_autovacuum/t/003_freeze_score.pl
new file mode 100644
index 00000000000..7769ea1fb07
--- /dev/null
+++ b/src/test/modules/test_autovacuum/t/003_freeze_score.pl
@@ -0,0 +1,85 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+# Raising an autovacuum freeze weight must never lower the corresponding
+# freeze score, for both the transaction ID and multixact components.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('main');
+$node->init;
+
+# Keep the *_freeze_max_age parameters well above the faked ages so the table
+# is not force-vacuumed for wraparound, and drop the failsafe ages to their
+# minimum so *_freeze_max_age alone governs the scaling threshold.
+$node->append_conf(
+ 'postgresql.conf', qq[
+autovacuum = off
+autovacuum_freeze_max_age = 1000000000
+autovacuum_multixact_freeze_max_age = 1000000000
+vacuum_failsafe_age = 0
+vacuum_multixact_failsafe_age = 0
+]);
+$node->start;
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE freezetest (i int) WITH (autovacuum_enabled = off)');
+
+# Fake old relfrozenxid/relminmxid rather than consuming hundreds of millions
+# of transactions. The score view only reads these fields arithmetically
+# (recentXid - relfrozenxid, recentMulti - relminmxid, both modulo 2^32) and
+# never consults clog for them, and with autovacuum off and the faked ages
+# below *_freeze_max_age nothing else acts on them, so this is safe. There is
+# no SQL primitive for the next multixact, but on a freshly initialized node it
+# is near the next xid, so subtracting from pg_snapshot_xmax() yields a usable
+# multixact age too; the assertions below confirm both ages landed in range.
+my $age = 300_000_000;
+$node->safe_psql(
+ 'postgres', qq[
+UPDATE pg_class
+ SET relfrozenxid = ((pg_snapshot_xmax(pg_current_snapshot())::text::bigint
+ - $age + 4294967296) % 4294967296)::text::xid,
+ relminmxid = ((pg_snapshot_xmax(pg_current_snapshot())::text::bigint
+ - $age + 4294967296) % 4294967296)::text::xid
+ WHERE relname = 'freezetest']);
+
+my ($xid_age, $mxid_age) = split /\|/, $node->safe_psql('postgres',
+ q[SELECT age(relfrozenxid) || '|' || mxid_age(relminmxid)
+ FROM pg_class WHERE relname = 'freezetest']);
+cmp_ok($xid_age, '>', 100_000_000, 'xid age reaches the scaled regime');
+cmp_ok($mxid_age, '>', 100_000_000, 'mxid age reaches the scaled regime');
+
+# Weights near 3.5 are where the unguarded code collapsed the score.
+my (%prev);
+foreach my $weight (1.0, 2.0, 3.0, 3.5, 5.0, 10.0)
+{
+ foreach my $kind (qw(freeze multixact_freeze))
+ {
+ $node->safe_psql('postgres',
+ "ALTER SYSTEM SET autovacuum_${kind}_score_weight = $weight");
+ }
+ $node->reload;
+ $node->poll_query_until('postgres',
+ "SELECT current_setting('autovacuum_freeze_score_weight')::float8 = $weight"
+ ) or die "timed out waiting for weight $weight";
+
+ my ($xid, $mxid) = split /\|/, $node->safe_psql('postgres',
+ q[SELECT xid_score || '|' || mxid_score
+ FROM pg_stat_autovacuum_scores WHERE relname = 'freezetest']);
+
+ foreach my $c (['xid', $xid], ['mxid', $mxid])
+ {
+ my ($name, $score) = @$c;
+ cmp_ok($score, '>', 0, "$name score positive at weight $weight");
+ cmp_ok($score, '>=', $prev{$name},
+ "$name score does not decrease at weight $weight")
+ if defined $prev{$name};
+ $prev{$name} = $score;
+ }
+}
+
+$node->stop;
+done_testing();
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-11-24 15:19 ` Sami Imseih <samimseih@gmail.com>
1 sibling, 0 replies; 150+ messages in thread
From: Sami Imseih @ 2025-11-24 15:19 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Nathan Bossart <nathandbossart@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> > What I have not been able to prove from my tests is that the processing
> > order of tables by autovacuum will actually make things any better or any
> > worse. My tests have been short 30 minute tests that count how many
> > vacuum cycles tables with various DML activity and sizes received.
> > I have not found much difference. I am also not sure how valuable
> > these short-duration tests are either.
>
> Yeah, I'm not sure that would be the right way to look for a benefit
> from something like this. I think that a better test scenario might
> involve figuring out how fast we can recover from a bad situation. As
> we've discussed before, if VACUUM is chronically unable to keep up
> with the workload, then the system is going to get into a very bad
> state and there's not really any help for it. But if we start to get
> into a bad situation due to some outside interference and then someone
> removes the interference, we might hope that this patch would help us
> get back on our feet more quickly.
>
> For instance, suppose that we have a database with a stale replication
> slot, so the oldest-XID value for the cluster keeps getting older and
> older. autovacuum is probably running but it can't clean anything up.
> Then at some point, the DBA realizes that bad things are happening and
> drops the replication slot. You might hope that, with the patch,
> autovacuum would do a better job getting the system back to a working
> state. If you set up some kind of test scenario, you could ask
> questions like "what is the largest age(relfrozenxid) that we observe
> in the database at any point during the test?" or "from the time the
> replication slot is dropped, how much time passes before
> age(datfrozenxid) drops to normal?" or "what is the maximum observed
> amount of bloat during the test?".
From my experience, in these situations, you need to run manual vacuums
to supplement autovacuum and get bloat under control as quickly as
possible. If the tables are small and vacuum quickly, the order of
prioritization doesn’t matter much, even with the extra bloat or high XID
age. However, if your slow-to-vacuum tables have the most bloat or the
oldest XID age, prioritizing those tables means that your smaller,
faster-to-vacuum
tables will almost certainly not get the vacuum cycles quickly enough
after resolving
whatever was blocking the vacuum (such as long-running transactions or stale
replication slots). In the current system, these smaller tables might still
get vacuumed, but often by pure chance due to the pg_class ordering.
Speeding up recovery (removing bloat and freezing rows) as soon as possible
will require enabling more autovacuum workers ( which are now dynamic )
or running manual vacuums. I don't think prioritization will improve these
situations much.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
@ 2025-11-22 20:03 ` Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 0 replies; 150+ messages in thread
From: Nathan Bossart @ 2025-11-22 20:03 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Robert Treat <rob@xzilla.net>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Sat, Nov 22, 2025 at 06:28:13AM -0500, Robert Haas wrote:
> What would be an issue is if we
> regressed some kind of common pattern. I admit that's a bit
> speculative and I'm probably being a little paranoid here: doing smart
> things is typically better than doing dumb things, and what we're
> doing right now is dumb.
>
> On the other hand, once we ship something, we can't pull it back. If
> it causes a problem, someone will call me at 2am and need their system
> fixed right now. If my answer is "well, there are no configuration
> knobs we can change and no way to get back to the old behavior and I'm
> sorry you're having that problem but the only answer is for you to run
> all your VACUUMs manually until two years from now when maybe the
> algorithm will have been improved," it's not going to be a very good
> night. After 15 years at EDB, I've learned that the problem isn't
> being wrong per se; it's having no way to get out from under being
> wrong.
Yeah. I'm tempted to code up the "weighting factor" GUCs for the next
revision. As you've noted, those would be useful for tuning and for
reverting back to pre-v19 behavior. Sure, we might end up with a handful
of retail GUCs that most users don't need, but that's not so terrible.
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Re: another autovacuum scheduling thread Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 14:30 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-11-22 13:07 ` Dilip Kumar <dilipbalaut@gmail.com>
1 sibling, 0 replies; 150+ messages in thread
From: Dilip Kumar @ 2025-11-22 13:07 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Sami Imseih <samimseih@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Thu, Nov 20, 2025 at 9:55 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
>
Thanks for working on this problem, We frequently hear about the auto
vacuuming scheduling issue. I believe this is a great starting point
to prioritize based on the wraparound and vacuum threshold limit.
However, my vision for addressing this problem has always involved
maintaining two distinct priority queues (or sorted lists). Each of
these queues would contain tables, with the tables within each queue
sorted by their respective scores.
Queue 1: Wraparound-Critical: This queue contains tables that require
immediate action because their XID or MultiXact ID age is critical,
especially those approaching the failsafe limit.
Queue 2: Threshold-Based: This queue includes tables needing VACUUM
due to crossing other thresholds.
Both queues would be maintained as sorted lists, with the highest
priority score at the head. The autovacuum worker dynamically selects
tables for processing from the head of these 2 queues. For instance,
if a table is initially chosen from the threshold queue but processing
took too long, and another table approaches its failsafe limit due to
a high rate of concurrent XID generation, the latter can be
prioritized from the wraparound queue. I believe this 2 queue
approach offers more flexibility than attempting to merge these
distinct concerns into a single scoring dimension.
Tables may exist in both queues. If a table is selected and vacuumed,
it will be removed from both queues to prevent redundant efforts.
--
Regards,
Dilip Kumar
Google
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-11 20:25 ` Sami Imseih <samimseih@gmail.com>
2025-11-11 20:43 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Sami Imseih @ 2025-11-11 20:25 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> On Sat, 8 Nov 2025 at 08:23, Sami Imseih <samimseih@gmail.com> wrote:
> > > I'm confused at why we'd have set up our autovacuum trigger points as
> > > they are today because we think those are good times to do a
> > > vacuum/analyze, but then prioritise on something completely different.
> > > Surely if we think 20% dead tuples is worth a vacuum, we must
> > > therefore think that 40% dead tuples are even more worthwhile?!
> >
> > Sure, but thresholds alone don't indicate anything about the how quick
> > the table can be vacuumed, # of indexes, per table a/v settings, etc.
> > The average a/v time is a good proxy to determine this.
> >
> > What I am suggesting here is we think beyond thresholds for
> > prioritization, and to give a chance for more eligible tables to get
> > autovacuumed rather than workers being saturated on some
> > of the slowest-to-vacuum tables.
>
> Can you define "more eligible" here?
What I mean by “more eligible” is that once a worker has its list of tables
that meet the autovacuum thresholds, it’s trying to get through as many
of them as possible within some time window.
If the workers always go after the slowest tables first, they’ll spend most
of that time on just a few heavy ones, and a lot of other eligible tables might
end up waiting much longer to get processed.
Eventually the slow tables will be the bottleneck anyway.
> I think I'm not really grasping this because I don't understand why
> faster-to-vacuum tables should be prioritised over slower-to-vacuum
> tables. Can you explain why you think this is important?
The thing I’m hoping to address is something I’ve seen many times in practice.
Autovacuum workers can get stuck on specific large or slow tables, and when
that happens, users often end up running manual vacuums on those tables
just to keep things moving for the smaller/faster vacuumed tables.
Now, I am not so sure any type of autovacuum prioritization could actually
help in these cases. What does help is adding more autovacuum workers.
> if we have the autovacuum worker refresh the list and scores after
> it's done with a table and autovacuum_naptime has elapsed since the
> list was last refreshed?
That is an interesting idea, but refreshing the list that often may not
be such a good idea, it could be quite expensive on large catalogs.
--
Sami Imseih
Amazon Web Services (AWS)
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:25 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
@ 2025-11-11 20:43 ` David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:53 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
0 siblings, 1 reply; 150+ messages in thread
From: David Rowley @ 2025-11-11 20:43 UTC (permalink / raw)
To: Sami Imseih <samimseih@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
On Wed, 12 Nov 2025 at 09:25, Sami Imseih <samimseih@gmail.com> wrote:
> The thing I’m hoping to address is something I’ve seen many times in practice.
> Autovacuum workers can get stuck on specific large or slow tables, and when
> that happens, users often end up running manual vacuums on those tables
> just to keep things moving for the smaller/faster vacuumed tables.
>
> Now, I am not so sure any type of autovacuum prioritization could actually
> help in these cases. What does help is adding more autovacuum workers.
Thanks for explaining. I think the scoring system in Nathan's patch
helps with this as any smaller table which are neglected continue to
bloat, and because they're smaller, the score will increase more
quickly, and eventually they'll have a higher score than the larger
tables. I think the situation you're talking about is when *all*
autovacuum workers are busy with large tables and no workers remaining
to deal with the now-higher-scoring smaller tables and they bloat
severely or statistics become wildly outdated as a result.
I'm aware of that problem. It seems to happen mostly when large tables
are busy receiving an anti-wraparound vacuum. I'm not sure what to do
about it, but I don't think changing the scoring system is the right
thing. Maybe we can have it configurable so that 1 worker can be
configured to not work on tables above a given size, so there's at
least 1 worker that is less likely to be tied up for extended periods
of time. I don't know if that's a good idea, and also don't know what
realistic values for "given size" are.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:47 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-29 15:51 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:25 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-11-11 20:43 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-11-11 20:53 ` Sami Imseih <samimseih@gmail.com>
0 siblings, 0 replies; 150+ messages in thread
From: Sami Imseih @ 2025-11-11 20:53 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; pgsql-hackers
> > The thing I’m hoping to address is something I’ve seen many times in practice.
> > Autovacuum workers can get stuck on specific large or slow tables, and when
> > that happens, users often end up running manual vacuums on those tables
> > just to keep things moving for the smaller/faster vacuumed tables.
> >
> > Now, I am not so sure any type of autovacuum prioritization could actually
> > help in these cases. What does help is adding more autovacuum workers.
>
> Thanks for explaining. I think the scoring system in Nathan's patch
> helps with this as any smaller table which are neglected continue to
> bloat, and because they're smaller, the score will increase more
> quickly,
That is true.
> Maybe we can have it configurable so that 1 worker can be
> configured to not work on tables above a given size, so there's at
> least 1 worker that is less likely to be tied up for extended periods
> of time. I don't know if that's a good idea, and also don't know what
> realistic values for "given size" are.
Another approach will be to signal for more autovacuum workers to
be spun up ( user can have a minimum and max workers ) if all workers
has been processing the list for a long time ( Also not sure what the
long threshold would be ). This "auto-tuning" of workers could perhaps
reduce the need for manual vacuums. It will still not prevent all workers
from being tied up, but maybe reduce the likelihood.
--
Sami
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
@ 2025-10-24 21:13 ` Peter Geoghegan <pg@bowt.ie>
2025-10-24 22:25 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Peter Geoghegan @ 2025-10-24 21:13 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Wed, Oct 22, 2025 at 3:35 PM David Rowley <dgrowleyml@gmail.com> wrote:
> If we had the varying sleep time as I mentioned above, the
> failsafe code could even be removed as the
> "autovacuum_vacuum_cost_delay / <tables score>" calculation would
> effectively zero the sleep time with any table > failsafe age.
I'm not sure what you mean by "the failsafe could be removed".
Importantly, the failsafe will abandon all further index vacuuming.
That's why it's presented as something that you as a user are not
supposed to rely on.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Re: another autovacuum scheduling thread Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Re: another autovacuum scheduling thread Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Re: another autovacuum scheduling thread Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` Re: another autovacuum scheduling thread David Rowley <dgrowleyml@gmail.com>
2025-10-24 21:13 ` Re: another autovacuum scheduling thread Peter Geoghegan <pg@bowt.ie>
@ 2025-10-24 22:25 ` David Rowley <dgrowleyml@gmail.com>
0 siblings, 0 replies; 150+ messages in thread
From: David Rowley @ 2025-10-24 22:25 UTC (permalink / raw)
To: Peter Geoghegan <pg@bowt.ie>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Sami Imseih <samimseih@gmail.com>; pgsql-hackers
On Sat, 25 Oct 2025 at 10:14, Peter Geoghegan <pg@bowt.ie> wrote:
>
> On Wed, Oct 22, 2025 at 3:35 PM David Rowley <dgrowleyml@gmail.com> wrote:
> > If we had the varying sleep time as I mentioned above, the
> > failsafe code could even be removed as the
> > "autovacuum_vacuum_cost_delay / <tables score>" calculation would
> > effectively zero the sleep time with any table > failsafe age.
>
> I'm not sure what you mean by "the failsafe could be removed".
> Importantly, the failsafe will abandon all further index vacuuming.
> That's why it's presented as something that you as a user are not
> supposed to rely on.
I didn't realise it did that too. I thought it just dropped the delay
to zero. In that case, I revoke the statement.
David
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-08 17:37 ` Andres Freund <andres@anarazel.de>
2025-10-09 16:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 150+ messages in thread
From: Andres Freund @ 2025-10-08 17:37 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers
Hi,
On 2025-10-08 10:18:17 -0500, Nathan Bossart wrote:
> However, we do no such prioritization of the tables within a database. In
> fact, the ordering of the tables is effectively random.
We don't prioritize tables, but I don't think the order really is random?
Isn't it basically in the order in which the data is in pg_class? That
typically won't change from one autovacuum pass to the next...
> * Prioritizing tables based on their (M)XID age might help avoid more
> aggressive vacuums, not to mention wraparound. Of course, there are
> scenarios where this doesn't work. For example, the age of a table may
> have changed greatly between the time we recorded it and the time we
> process it.
> Or maybe there is another table in a different database that
> is more important from a wraparound perspective.
That seems like something no ordering within a single AV worker can address. I
think it's fine to just define that to be out of scope.
> We could complicate the patch to try to handle some of these things, but I
> maintain that even some basic, incremental scheduling improvements would be
> better than the status quo. And we can always change it further in the
> future to handle these problems and to consider other things like bloat.
Agreed! It doesn't take much to be better at scheduling than "order in
pg_class".
> The attached patch works by storing the maximum of the XID age and the MXID
> age in the list with the OIDs and sorting it prior to processing.
I think it may be worth trying to avoid reliably using the same order -
otherwise e.g. a corrupt index on the first scheduled table can cause
autovacuum to reliably fail on the same relation, never allowing it to
progress past that point.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:37 ` Re: another autovacuum scheduling thread Andres Freund <andres@anarazel.de>
@ 2025-10-09 16:01 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-09 16:15 ` Re: another autovacuum scheduling thread Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 150+ messages in thread
From: Nathan Bossart @ 2025-10-09 16:01 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers
On Wed, Oct 08, 2025 at 01:37:22PM -0400, Andres Freund wrote:
> On 2025-10-08 10:18:17 -0500, Nathan Bossart wrote:
>> The attached patch works by storing the maximum of the XID age and the MXID
>> age in the list with the OIDs and sorting it prior to processing.
>
> I think it may be worth trying to avoid reliably using the same order -
> otherwise e.g. a corrupt index on the first scheduled table can cause
> autovacuum to reliably fail on the same relation, never allowing it to
> progress past that point.
Hm. What if we kept a short array of "failed" tables in shared memory?
Each worker would consult this table before processing. If the table is
there, it would remove it from the shared table and skip processing it.
Then the next worker would try processing the table again.
I also wonder how hard it would be to gracefully catch the error and let
the worker continue with the rest of its list...
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:37 ` Re: another autovacuum scheduling thread Andres Freund <andres@anarazel.de>
2025-10-09 16:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
@ 2025-10-09 16:15 ` Andres Freund <andres@anarazel.de>
2025-10-09 16:33 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-09 19:45 ` Re: another autovacuum scheduling thread Peter Geoghegan <pg@bowt.ie>
0 siblings, 2 replies; 150+ messages in thread
From: Andres Freund @ 2025-10-09 16:15 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers
Hi,
On 2025-10-09 11:01:16 -0500, Nathan Bossart wrote:
> On Wed, Oct 08, 2025 at 01:37:22PM -0400, Andres Freund wrote:
> > On 2025-10-08 10:18:17 -0500, Nathan Bossart wrote:
> >> The attached patch works by storing the maximum of the XID age and the MXID
> >> age in the list with the OIDs and sorting it prior to processing.
> >
> > I think it may be worth trying to avoid reliably using the same order -
> > otherwise e.g. a corrupt index on the first scheduled table can cause
> > autovacuum to reliably fail on the same relation, never allowing it to
> > progress past that point.
>
> Hm. What if we kept a short array of "failed" tables in shared memory?
I've thought about having that as part of pgstats...
> Each worker would consult this table before processing. If the table is
> there, it would remove it from the shared table and skip processing it.
> Then the next worker would try processing the table again.
>
> I also wonder how hard it would be to gracefully catch the error and let
> the worker continue with the rest of its list...
The main set of cases I've seen are when workers get hung up permanently in
corrupt indexes. There never is actually an error, the autovacuums just get
terminated as part of whatever independent reason there is to restart. The
problem with that is that you'll never actually have vacuum fail...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:37 ` Re: another autovacuum scheduling thread Andres Freund <andres@anarazel.de>
2025-10-09 16:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-09 16:15 ` Re: another autovacuum scheduling thread Andres Freund <andres@anarazel.de>
@ 2025-10-09 16:33 ` Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 0 replies; 150+ messages in thread
From: Nathan Bossart @ 2025-10-09 16:33 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers
On Thu, Oct 09, 2025 at 12:15:31PM -0400, Andres Freund wrote:
> On 2025-10-09 11:01:16 -0500, Nathan Bossart wrote:
>> I also wonder how hard it would be to gracefully catch the error and let
>> the worker continue with the rest of its list...
>
> The main set of cases I've seen are when workers get hung up permanently in
> corrupt indexes. There never is actually an error, the autovacuums just get
> terminated as part of whatever independent reason there is to restart. The
> problem with that is that you'll never actually have vacuum fail...
Ah. Wouldn't the other workers skip that table in that scenario? I'm not
following the great advantage of varying the order in this case. I suppose
the full set of workers might be able to process more tables before one
inevitably gets stuck. Is that it?
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
* Re: another autovacuum scheduling thread
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:37 ` Re: another autovacuum scheduling thread Andres Freund <andres@anarazel.de>
2025-10-09 16:01 ` Re: another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-09 16:15 ` Re: another autovacuum scheduling thread Andres Freund <andres@anarazel.de>
@ 2025-10-09 19:45 ` Peter Geoghegan <pg@bowt.ie>
1 sibling, 0 replies; 150+ messages in thread
From: Peter Geoghegan @ 2025-10-09 19:45 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Thu, Oct 9, 2025 at 12:15 PM Andres Freund <andres@anarazel.de> wrote:
> > Each worker would consult this table before processing. If the table is
> > there, it would remove it from the shared table and skip processing it.
> > Then the next worker would try processing the table again.
> >
> > I also wonder how hard it would be to gracefully catch the error and let
> > the worker continue with the rest of its list...
>
> The main set of cases I've seen are when workers get hung up permanently in
> corrupt indexes.
How recently was this? I'm aware of problems like that that we
discussed around 2018, but they were greatly mitigated.
First by your commit 3a01f68e, then by my commit c34787f9.
In general, there's no particularly good reason why (at least with
nbtree indexes) VACUUM should ever hang forever. The access pattern is
overwhelmingly simple, sequential access. The only exception is nbtree
page deletion (plus backtracking), where it isn't particularly hard to
just be very careful about self-deadlock.
> There never is actually an error, the autovacuums just get
> terminated as part of whatever independent reason there is to restart.
What do you mean?
In general I'd expect nbtree VACUUM of a corrupt index to either not
fail at all (we'll soldier on to the best of our ability when page
deletion encounters an inconsistency), or to get permanently stuck due
to locking the same page twice/self-deadlock (though as I said, those
problems were mitigated, and might even be almost impossible these
days). Every other case involves some kind of error (e.g., an OOM is
just about possible).
I agree with you about using a perfectly deterministic order coming
with real downsides, without any upside. Don't interpret what I've
said as expressing opposition to that idea.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 150+ messages in thread
* 回复：回复：another autovacuum scheduling thread
@ 2025-11-14 02:25 =?UTF-8?B?5q615Z2k5LuBKOWIu+mfpyk=?= <duankunren.dkr@alibaba-inc.com>
0 siblings, 0 replies; 150+ messages in thread
From: 段坤仁(刻韧) @ 2025-11-14 02:25 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Sami Imseih <samimseih@gmail.com>; Jeremy Schneider <schneider@ardentperf.com>; Robert Treat <rob@xzilla.net>; David Rowley <dgrowleyml@gmail.com>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers
Thank you for your reply. You are absolutely right, and I apologize for straying off-topic in this thread. I have moved my thoughts to a separate thread [0].
[0]https://www.postgresql.org/message-id/fffdd62e-4d97-4701-ad57-0cd3ef1ebef4.duankunren.dkr%40alibaba-... <https://www.postgresql.org/message-id/fffdd62e-4d97-4701-ad57-0cd3ef1ebef4.duankunren.dkr%40alibaba-... >
------------------------------------------------------------------
发件人：Nathan Bossart <nathandbossart@gmail.com>
发送时间：2025年11月14日(周五) 02:47
收件人："段坤仁(刻韧)"<duankunren.dkr@alibaba-inc.com>
抄 送：Sami Imseih<samimseih@gmail.com>; Jeremy Schneider<schneider@ardentperf.com>; Robert Treat<rob@xzilla.net>; David Rowley<dgrowleyml@gmail.com>; Robert Haas<robertmhaas@gmail.com>; "pgsql-hackers"<pgsql-hackers@postgresql.org>
主 题：Re: 回复：another autovacuum scheduling thread
On Fri, Nov 14, 2025 at 02:37:31AM +0800, 段坤仁(刻韧) wrote:
> Hi all, I have read the discussion in this thread and am pleased to see the
> community working collaboratively to address some long-standing autovacuum
> problems. Nathan's patch implementation is quite promising and demonstrates
> considerable potential. I have previously attempted similar approaches; however,
> I was unable to develop such a comprehensive and well-formulated calculation
> framework. We would certainly welcome the integration of this improvement into
> v19 or subsequent versions.
Thanks!
> I have tried Nathan's V7 patch and implemented some cost delay mechanisms
> based on it that might be helpful for the issues you guys mentioned.
Unfortunately, cost delay adjustments are off-topic for this thread, as I
hinted yesterday [0]. I'd certainly like to explore this idea, but if we
can't keep the discussion focused, it'll be hard to get anything committed.
[0] https://postgr.es/m/aRTpqMleDpoQm9OO%40nathan <https://postgr.es/m/aRTpqMleDpoQm9OO%40nathan >
--
nathan
^ permalink raw reply [nested|flat] 150+ messages in thread
end of thread, other threads:[~2026-09-02 01:19 UTC | newest]
Thread overview: 150+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-10-08 15:18 another autovacuum scheduling thread Nathan Bossart <nathandbossart@gmail.com>
2025-10-08 17:06 ` Sami Imseih <samimseih@gmail.com>
2025-10-08 17:20 ` Álvaro Herrera <alvherre@kurilemu.de>
2025-10-08 17:47 ` Sami Imseih <samimseih@gmail.com>
2025-10-08 23:40 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-08 23:59 ` David Rowley <dgrowleyml@gmail.com>
2025-10-09 00:27 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 00:30 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:03 ` David Rowley <dgrowleyml@gmail.com>
2025-10-09 01:25 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 01:47 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-09 03:13 ` David Rowley <dgrowleyml@gmail.com>
2025-10-09 16:13 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 17:31 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 18:42 ` Robert Haas <robertmhaas@gmail.com>
2025-10-10 19:44 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-10 20:24 ` Robert Haas <robertmhaas@gmail.com>
2025-10-10 21:59 ` Jeremy Schneider <schneider@ardentperf.com>
2025-10-13 12:32 ` Robert Haas <robertmhaas@gmail.com>
2025-10-12 06:27 ` David Rowley <dgrowleyml@gmail.com>
2025-10-21 14:38 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-21 20:07 ` David Rowley <dgrowleyml@gmail.com>
2025-10-22 18:40 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 18:58 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-22 19:34 ` David Rowley <dgrowleyml@gmail.com>
2025-10-22 19:43 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 18:22 ` Sami Imseih <samimseih@gmail.com>
2025-10-23 18:47 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-23 19:32 ` Sami Imseih <samimseih@gmail.com>
2025-10-23 20:24 ` David Rowley <dgrowleyml@gmail.com>
2025-10-23 20:48 ` Sami Imseih <samimseih@gmail.com>
2025-10-23 22:39 ` David Rowley <dgrowleyml@gmail.com>
2025-10-24 15:08 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-26 01:25 ` David Rowley <dgrowleyml@gmail.com>
2025-10-27 16:06 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 17:47 ` Sami Imseih <samimseih@gmail.com>
2025-10-27 21:15 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-27 22:35 ` Sami Imseih <samimseih@gmail.com>
2025-10-27 23:16 ` David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:06 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-29 15:24 ` Sami Imseih <samimseih@gmail.com>
2025-10-29 16:07 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 02:58 ` wenhui qiu <qiuwenhuifx@gmail.com>
2025-10-30 03:41 ` David Rowley <dgrowleyml@gmail.com>
2025-10-30 06:48 ` wenhui qiu <qiuwenhuifx@gmail.com>
2025-10-30 10:36 ` David Rowley <dgrowleyml@gmail.com>
2025-10-27 22:47 ` David Rowley <dgrowleyml@gmail.com>
2025-10-28 21:03 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-28 22:44 ` Sami Imseih <samimseih@gmail.com>
2025-10-29 03:10 ` wenhui qiu <qiuwenhuifx@gmail.com>
2025-10-29 15:58 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-29 15:51 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 20:05 ` Robert Haas <robertmhaas@gmail.com>
2025-10-30 21:02 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-30 21:05 ` Sami Imseih <samimseih@gmail.com>
2025-10-31 00:38 ` Sami Imseih <samimseih@gmail.com>
2025-10-31 20:12 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-01 01:50 ` David Rowley <dgrowleyml@gmail.com>
2025-11-01 03:29 ` David Rowley <dgrowleyml@gmail.com>
2025-11-06 22:21 ` Sami Imseih <samimseih@gmail.com>
2025-11-06 23:05 ` David Rowley <dgrowleyml@gmail.com>
2025-11-07 19:22 ` Sami Imseih <samimseih@gmail.com>
2025-11-11 00:58 ` David Rowley <dgrowleyml@gmail.com>
2025-11-11 16:36 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 19:43 ` Robert Treat <rob@xzilla.net>
2025-11-11 19:48 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 19:50 ` Robert Treat <rob@xzilla.net>
2025-11-11 20:16 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:03 ` David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:13 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-11 20:26 ` David Rowley <dgrowleyml@gmail.com>
2025-11-11 23:22 ` Robert Treat <rob@xzilla.net>
2025-11-12 20:10 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-12 22:10 ` Sami Imseih <samimseih@gmail.com>
2025-11-12 23:51 ` Jeremy Schneider <schneider@ardentperf.com>
2025-11-13 00:32 ` Sami Imseih <samimseih@gmail.com>
2025-11-20 14:30 ` Robert Haas <robertmhaas@gmail.com>
2025-11-20 16:25 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-20 16:34 ` Sami Imseih <samimseih@gmail.com>
2025-11-20 18:35 ` Robert Haas <robertmhaas@gmail.com>
2025-11-20 20:21 ` Sami Imseih <samimseih@gmail.com>
2025-11-20 20:58 ` David Rowley <dgrowleyml@gmail.com>
2025-11-20 21:16 ` Robert Haas <robertmhaas@gmail.com>
2025-11-20 22:12 ` David Rowley <dgrowleyml@gmail.com>
2025-11-22 11:28 ` Robert Haas <robertmhaas@gmail.com>
2025-11-22 17:28 ` Sami Imseih <samimseih@gmail.com>
2025-11-22 18:35 ` Robert Haas <robertmhaas@gmail.com>
2025-11-23 09:55 ` David Rowley <dgrowleyml@gmail.com>
2025-11-24 19:59 ` Robert Haas <robertmhaas@gmail.com>
2026-03-05 17:03 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 15:06 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-10 16:19 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 00:11 ` Sami Imseih <samimseih@gmail.com>
2026-03-11 01:08 ` Sami Imseih <samimseih@gmail.com>
2026-03-11 03:56 ` wenhui qiu <qiuwenhuifx@gmail.com>
2026-03-11 15:53 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:08 ` Sami Imseih <samimseih@gmail.com>
2026-03-11 17:28 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-11 17:59 ` Sami Imseih <samimseih@gmail.com>
2026-03-12 19:20 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-17 23:06 ` David Rowley <dgrowleyml@gmail.com>
2026-03-18 16:09 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 09:55 ` Greg Burd <greg@burd.me>
2026-03-19 11:44 ` David Rowley <dgrowleyml@gmail.com>
2026-03-19 13:49 ` Greg Burd <greg@burd.me>
2026-03-19 15:49 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-19 17:36 ` Greg Burd <greg@burd.me>
2026-03-20 04:22 ` Sami Imseih <samimseih@gmail.com>
2026-03-20 04:40 ` David Rowley <dgrowleyml@gmail.com>
2026-03-20 05:03 ` Sami Imseih <samimseih@gmail.com>
2026-03-20 20:31 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-21 14:55 ` Sami Imseih <samimseih@gmail.com>
2026-03-22 00:08 ` Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-23 15:22 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 19:01 ` Sami Imseih <samimseih@gmail.com>
2026-03-23 21:01 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-23 22:27 ` Jim Nasby <jnasby@upgrade.com>
2026-03-24 00:32 ` David Rowley <dgrowleyml@gmail.com>
2026-03-24 17:15 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-24 21:12 ` Sami Imseih <samimseih@gmail.com>
2026-03-25 21:12 ` Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 21:18 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-25 21:24 ` Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-03-25 22:00 ` David Rowley <dgrowleyml@gmail.com>
2026-03-25 22:10 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 00:28 ` David Rowley <dgrowleyml@gmail.com>
2026-03-26 16:49 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-26 21:29 ` David Rowley <dgrowleyml@gmail.com>
2026-03-27 15:18 ` Nathan Bossart <nathandbossart@gmail.com>
2026-03-27 23:17 ` Sami Imseih <samimseih@gmail.com>
2026-08-31 15:23 ` Nathan Bossart <nathandbossart@gmail.com>
2026-08-31 15:49 ` Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2026-08-31 16:47 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-01 05:18 ` Greg Burd <greg@burd.me>
2026-09-01 20:22 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-02 01:19 ` shihao zhong <zhong950419@gmail.com>
2025-11-24 15:19 ` Sami Imseih <samimseih@gmail.com>
2025-11-22 20:03 ` Nathan Bossart <nathandbossart@gmail.com>
2025-11-22 13:07 ` Dilip Kumar <dilipbalaut@gmail.com>
2025-11-11 20:25 ` Sami Imseih <samimseih@gmail.com>
2025-11-11 20:43 ` David Rowley <dgrowleyml@gmail.com>
2025-11-11 20:53 ` Sami Imseih <samimseih@gmail.com>
2025-10-24 21:13 ` Peter Geoghegan <pg@bowt.ie>
2025-10-24 22:25 ` David Rowley <dgrowleyml@gmail.com>
2025-10-08 17:37 ` Andres Freund <andres@anarazel.de>
2025-10-09 16:01 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-09 16:15 ` Andres Freund <andres@anarazel.de>
2025-10-09 16:33 ` Nathan Bossart <nathandbossart@gmail.com>
2025-10-09 19:45 ` Peter Geoghegan <pg@bowt.ie>
2025-11-14 02:25 回复：回复：another autovacuum scheduling thread =?UTF-8?B?5q615Z2k5LuBKOWIu+mfpyk=?= <duankunren.dkr@alibaba-inc.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox